Compare commits

..

12 Commits

Author SHA1 Message Date
352c93b680 manifest: add support for groups in include
Attrib groups can now be added to manifest include, thus
all projects in an included manifest file can easily be tagged
with a group without modifying all projects in that manifest file.

Include groups will add and recurse, meaning included manifest
projects will carry all parent includes. Intentionally, no support
added for group remove, to keep complexity down.

Group handling for projects is untouched, meaning a group set on
a project will still append to whatever was or was not inherited
in parent manifest includes, resulting in union of groups inherited
and set for the project itself.

Test: manual multi-level manifest include structure, in serial and parallel,
      with different groups set on init
Test: added unit tests to cover the inheritance

Change-Id: Id2229aa6fd78d355ba598cc15c701b2ee71e5c6f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/283587
Tested-by: Fredrik de Groot <fredrik.de.groot@volvocars.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-11-26 09:13:14 +00:00
7f7acfe9fd Concentrate the RepoHook knowledge in the RepoHook class
The knowledge about running hooks and all its exception handling
is scattered over multiple files. This makes the code harder
to read, but also it requires duplication of logic in case
other RepoHooks are added to different commands.
This refactoring also creates uniform behavior of the hooks
across multiple commands and it guarantees the re-use of the same
arguments on all of them.

Signed-off-by: Remy Bohmer <github@bohmer.net>
Change-Id: Ia4d90eab429e4af00943306e89faec8db35ba29d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/277562
Tested-by: Remy Bohmer <oss@bohmer.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-11-23 09:59:16 +00:00
169b0218b3 Fix --reference option under Windows
When intializing a new repo with the --reference option on Windows 10
the objects/info/alternates in each git repository is created with
Windows line endings (\r\n), leading to the following error:

error: object directory C:/<PATH_TO_MIRROR>/<REPO_NAME>.git/objects?
does not exist; check .git/objects/info/alternates

This can be fixed by simply using unix line endings on both
Windows and unix platforms.

Reported-by: Francisco Javier Alvarez Garcia <javier.alvarez.garcia.17@gmail.com>
Follow-up-from: I268fe029ede68802c21037b0f2ae8a95afb85e48
Bug: https://crbug.com/gerrit/13208
Change-Id: I6da60c4ca957778b3c42ab6b9ad85c40483f0042
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/289431
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Remy Bohmer <oss@bohmer.net>
2020-11-23 09:17:32 +00:00
44bc9643ed Always use Unix EOL for worktree .git and gitdir files
Worktree .git and gitdir reference files are written by Git with
Unix line ending, even on Windows & macOS. The conversion to
relative paths makes these files end with DOS line endings in
Windows.  The Git integration in Visual Studio 2019 cannot deal
with these DOS line endings and considers these worktrees invalid.

Signed-off-by: Remy Bohmer <github@bohmer.net>
Change-Id: I088cfd994f3cc31db4e0ca7791fa0a4ee3ac222f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/289310
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Remy Bohmer <linux@bohmer.net>
2020-11-20 20:53:43 +00:00
d7f8683daf project: do not update local published/ refs in dryrun mode
Bug: https://crbug.com/gerrit/13087
Change-Id: I197e6d6d07c7d325ac294b597d42e895f77c737f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/289182
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-20 04:08:19 +00:00
8c1e9cbef1 manifest_xml: refactor manifest parsing from client management
We conflate the manifest & parsing logic with the management of the
repo client checkout in a single class.  This makes testing just one
part (the manifest parsing) hard as it requires a full checkout too.

Start splitting the two apart into separate classes to make it easy
to reason about & test.

Change-Id: Iaf897c93db9c724baba6044bfe7a589c024523b2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288682
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-18 19:10:57 +00:00
a488af5ea5 main: require Python 3 now
We've been warning about this for more than 6 months (with public
announcements even older).  Lets make it a failure now to see who
hasn't upgraded yet.

Bug: https://crbug.com/gerrit/10418
Change-Id: Iec3e2cbf87de434021921616683d360bc4fef77a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280796
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-17 15:04:20 +00:00
e283b95cf2 tests: use new main branch
Now that we clone "main" by default, use that for our local test.

Change-Id: Ib8420074bdfabfcb9d5252a3a0ecd3d852ca36e8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288422
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-17 04:29:09 +00:00
dc5c4d1d11 sync: respect --force-sync when fetching manifest project updates
The --force-sync option was being passed down for all updates except
for the manifest project, so add that there too.

Bug: https://crbug.com/gerrit/11034
Change-Id: I33818b652f828c6b847dbc70f1fedfac5ac17bbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/228146
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-11-17 03:06:06 +00:00
23411d3f9c manifest: add a --json output option
Sometimes parsing JSON is easier than parsing XML, especially when
the XML format is limited (which ours is).  Add a --json option to
the manifest command to quickly emit that form.

Bug: https://crbug.com/gerrit/11743
Change-Id: Ia2bb254a78ae2b70a851638b4545fcafe8c1a76b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280436
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-17 01:38:00 +00:00
160748f828 upload: improve tip for fixing upload remote
Instead of assuming the repo client is tracking the "master" branch
of the manifest repo, use the existing info we have to display the
right info to the user.

Bug: https://crbug.com/gerrit/13339
Change-Id: I8b265f4b2e075fdc41909b1f3dff9aee87384353
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/287279
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-16 23:13:02 +00:00
6e89c965f4 switch to "main" branch for development
We're migrating from "master" to "main" as the default development
branch.  This only affects repo itself, not manifests.

Change-Id: I27489dd721c9a467a1c43736808cb3b3c1365433
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288082
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-11-16 05:07:33 +00:00
25 changed files with 475 additions and 200 deletions

View File

@ -5,7 +5,7 @@ name: Test CI
on: on:
push: push:
branches: [master, repo-1, stable, maint] branches: [main, repo-1, stable, maint]
tags: [v*] tags: [v*]
jobs: jobs:

View File

@ -1,8 +1,5 @@
# repo # repo
> **Warning: The "master" branch is no longer used. Use "main" instead.**<br>
> https://gerrit.googlesource.com/git-repo/+/HEAD/README.md
Repo is a tool built on top of Git. Repo helps manage many Git repositories, Repo is a tool built on top of Git. Repo helps manage many Git repositories,
does the uploads to revision control systems, and automates parts of the does the uploads to revision control systems, and automates parts of the
development workflow. Repo is not meant to replace Git, only to make it development workflow. Repo is not meant to replace Git, only to make it

View File

@ -1,6 +1,3 @@
> **Warning: The "master" branch is no longer used. Use "main" instead.**<br>
> https://gerrit.googlesource.com/git-repo/+/HEAD/SUBMITTING_PATCHES.md
[TOC] [TOC]
# Short Version # Short Version
@ -13,7 +10,7 @@
- Make corrections if requested. - Make corrections if requested.
- Verify your changes on gerrit so they can be submitted. - Verify your changes on gerrit so they can be submitted.
`git push https://gerrit-review.googlesource.com/git-repo HEAD:refs/for/master` `git push https://gerrit-review.googlesource.com/git-repo HEAD:refs/for/main`
# Long Version # Long Version
@ -153,7 +150,7 @@ Push your patches over HTTPS to the review server, possibly through
a remembered remote to make this easier in the future: a remembered remote to make this easier in the future:
git config remote.review.url https://gerrit-review.googlesource.com/git-repo git config remote.review.url https://gerrit-review.googlesource.com/git-repo
git config remote.review.push HEAD:refs/for/master git config remote.review.push HEAD:refs/for/main
git push review git push review

View File

@ -1,8 +1,5 @@
# Repo internal filesystem layout # Repo internal filesystem layout
> **Warning: The "master" branch is no longer used. Use "main" instead.**<br>
> https://gerrit.googlesource.com/git-repo/+/HEAD/docs/internal-fs-layout.md
A reference to the `.repo/` tree in repo client checkouts. A reference to the `.repo/` tree in repo client checkouts.
Hopefully it's complete & up-to-date, but who knows! Hopefully it's complete & up-to-date, but who knows!
@ -109,7 +106,7 @@ support, see the [manifest-format.md] file.
setting in the manifest (i.e. the path on the remote server) with a `.git` setting in the manifest (i.e. the path on the remote server) with a `.git`
suffix. This allows for multiple checkouts of the same remote git repo to suffix. This allows for multiple checkouts of the same remote git repo to
share their objects. For example, you could have different branches of share their objects. For example, you could have different branches of
`foo/bar.git` checked out to `foo/bar-master`, `foo/bar-release`, etc... `foo/bar.git` checked out to `foo/bar-main`, `foo/bar-release`, etc...
There will be multiple trees under `projects/` for each one, but only one There will be multiple trees under `projects/` for each one, but only one
under `project-objects/`. under `project-objects/`.

View File

@ -1,8 +1,5 @@
# repo Manifest Format # repo Manifest Format
> **Warning: The "master" branch is no longer used. Use "main" instead.**<br>
> https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
A repo manifest describes the structure of a repo client; that is A repo manifest describes the structure of a repo client; that is
the directories that are visible and where they should be obtained the directories that are visible and where they should be obtained
from with git. from with git.
@ -103,6 +100,7 @@ following DTD:
<!ELEMENT include EMPTY> <!ELEMENT include EMPTY>
<!ATTLIST include name CDATA #REQUIRED> <!ATTLIST include name CDATA #REQUIRED>
<!ATTLIST include groups CDATA #IMPLIED>
]> ]>
``` ```
@ -145,8 +143,8 @@ Attribute `review`: Hostname of the Gerrit server where reviews
are uploaded to by `repo upload`. This attribute is optional; are uploaded to by `repo upload`. This attribute is optional;
if not specified then `repo upload` will not function. if not specified then `repo upload` will not function.
Attribute `revision`: Name of a Git branch (e.g. `master` or Attribute `revision`: Name of a Git branch (e.g. `main` or
`refs/heads/master`). Remotes with their own revision will override `refs/heads/main`). Remotes with their own revision will override
the default revision. the default revision.
### Element default ### Element default
@ -159,11 +157,11 @@ Attribute `remote`: Name of a previously defined remote element.
Project elements lacking a remote attribute of their own will use Project elements lacking a remote attribute of their own will use
this remote. this remote.
Attribute `revision`: Name of a Git branch (e.g. `master` or Attribute `revision`: Name of a Git branch (e.g. `main` or
`refs/heads/master`). Project elements lacking their own `refs/heads/main`). Project elements lacking their own
revision attribute will use this revision. revision attribute will use this revision.
Attribute `dest-branch`: Name of a Git branch (e.g. `master`). Attribute `dest-branch`: Name of a Git branch (e.g. `main`).
Project elements not setting their own `dest-branch` will inherit Project elements not setting their own `dest-branch` will inherit
this value. If this value is not set, projects will use `revision` this value. If this value is not set, projects will use `revision`
by default instead. by default instead.
@ -250,13 +248,13 @@ If not supplied the remote given by the default element is used.
Attribute `revision`: Name of the Git branch the manifest wants Attribute `revision`: Name of the Git branch the manifest wants
to track for this project. Names can be relative to refs/heads to track for this project. Names can be relative to refs/heads
(e.g. just "master") or absolute (e.g. "refs/heads/master"). (e.g. just "main") or absolute (e.g. "refs/heads/main").
Tags and/or explicit SHA-1s should work in theory, but have not Tags and/or explicit SHA-1s should work in theory, but have not
been extensively tested. If not supplied the revision given by been extensively tested. If not supplied the revision given by
the remote element is used if applicable, else the default the remote element is used if applicable, else the default
element is used. element is used.
Attribute `dest-branch`: Name of a Git branch (e.g. `master`). Attribute `dest-branch`: Name of a Git branch (e.g. `main`).
When using `repo upload`, changes will be submitted for code When using `repo upload`, changes will be submitted for code
review on this branch. If unspecified both here and in the review on this branch. If unspecified both here and in the
default element, `revision` is used instead. default element, `revision` is used instead.
@ -371,6 +369,10 @@ target manifest to include - it must be a usable manifest on its own.
Attribute `name`: the manifest to include, specified relative to Attribute `name`: the manifest to include, specified relative to
the manifest repository's root. the manifest repository's root.
Attribute `groups`: List of additional groups to which all projects
in the included manifest belong. This appends and recurses, meaning
all projects in sub-manifests carry all parent include groups.
Same syntax as the corresponding element of `project`.
## Local Manifests ## Local Manifests

View File

@ -1,8 +1,5 @@
# Supported Python Versions # Supported Python Versions
> **Warning: The "master" branch is no longer used. Use "main" instead.**<br>
> https://gerrit.googlesource.com/git-repo/+/HEAD/docs/python-support.md
With Python 2.7 officially going EOL on [01 Jan 2020](https://pythonclock.org/), With Python 2.7 officially going EOL on [01 Jan 2020](https://pythonclock.org/),
we need a support plan for the repo project itself. we need a support plan for the repo project itself.
Inevitably, there will be a long tail of users who still want to use Python 2 on Inevitably, there will be a long tail of users who still want to use Python 2 on
@ -21,13 +18,13 @@ Bugfixes may be added on a best-effort basis or from the community, but largely
no new features will be added, nor is support guaranteed. no new features will be added, nor is support guaranteed.
Users can select this during `repo init` time via the [repo launcher]. Users can select this during `repo init` time via the [repo launcher].
Otherwise the default branches (e.g. stable & master) will be used which will Otherwise the default branches (e.g. stable & main) will be used which will
require Python 3. require Python 3.
This means the [repo launcher] needs to support both Python 2 & Python 3, but This means the [repo launcher] needs to support both Python 2 & Python 3, but
since it doesn't import any other repo code, this shouldn't be too problematic. since it doesn't import any other repo code, this shouldn't be too problematic.
The master branch will require Python 3.6 at a minimum. The main branch will require Python 3.6 at a minimum.
If the system has an older version of Python 3, then users will have to select If the system has an older version of Python 3, then users will have to select
the legacy Python 2 branch instead. the legacy Python 2 branch instead.

View File

@ -1,8 +1,5 @@
# repo release process # repo release process
> **Warning: The "master" branch is no longer used. Use "main" instead.**<br>
> https://gerrit.googlesource.com/git-repo/+/HEAD/docs/release-process.md
This is the process for creating a new release of repo, as well as all the This is the process for creating a new release of repo, as well as all the
related topics and flows. related topics and flows.
@ -100,7 +97,7 @@ If that tag cannot be verified, it gives up and forces the user to resolve.
## Branch management ## Branch management
All development happens on the `master` branch and should generally be stable. All development happens on the `main` branch and should generally be stable.
Since the repo launcher defaults to tracking the `stable` branch, it is not Since the repo launcher defaults to tracking the `stable` branch, it is not
normally updated until a new release is available. normally updated until a new release is available.
@ -115,7 +112,7 @@ For example, when `stable` moves from `v1.10.x` to `v1.11.x`, then the `maint`
branch will be updated from `v1.9.x` to `v1.10.x`. branch will be updated from `v1.9.x` to `v1.10.x`.
We don't have parallel release branches/series. We don't have parallel release branches/series.
Typically all tags are made against the `master` branch and then pushed to the Typically all tags are made against the `main` branch and then pushed to the
`stable` branch to make it available to the rest of the world. `stable` branch to make it available to the rest of the world.
Since repo doesn't typically see a lot of changes, this tends to be OK. Since repo doesn't typically see a lot of changes, this tends to be OK.
@ -123,10 +120,10 @@ Since repo doesn't typically see a lot of changes, this tends to be OK.
When you want to create a new release, you'll need to select a good version and When you want to create a new release, you'll need to select a good version and
create a signed tag using a key registered in repo itself. create a signed tag using a key registered in repo itself.
Typically we just tag the latest version of the `master` branch. Typically we just tag the latest version of the `main` branch.
The tag could be pushed now, but it won't be used by clients normally (since the The tag could be pushed now, but it won't be used by clients normally (since the
default `repo-rev` setting is `stable`). default `repo-rev` setting is `stable`).
This would allow some early testing on systems who explicitly select `master`. This would allow some early testing on systems who explicitly select `main`.
### Creating a signed tag ### Creating a signed tag
@ -147,7 +144,7 @@ $ export GNUPGHOME=~/.gnupg/repo/
$ gpg -K $ gpg -K
# Pick whatever branch or commit you want to tag. # Pick whatever branch or commit you want to tag.
$ r=master $ r=main
# Pick the new version. # Pick the new version.
$ t=1.12.10 $ t=1.12.10

View File

@ -1,8 +1,5 @@
# repo hooks # repo hooks
> **Warning: The "master" branch is no longer used. Use "main" instead.**<br>
> https://gerrit.googlesource.com/git-repo/+/HEAD/docs/repo-hooks.md
[TOC] [TOC]
Repo provides a mechanism to hook specific stages of the runtime with custom Repo provides a mechanism to hook specific stages of the runtime with custom
@ -30,7 +27,7 @@ repohooks project is updated and a hook is triggered.
For the full syntax, see the [repo manifest format](./manifest-format.md). For the full syntax, see the [repo manifest format](./manifest-format.md).
Here's a short example from Here's a short example from
[Android](https://android.googlesource.com/platform/manifest/+/master/default.xml). [Android](https://android.googlesource.com/platform/manifest/+/HEAD/default.xml).
The `<project>` line checks out the repohooks git repo to the local The `<project>` line checks out the repohooks git repo to the local
`tools/repohooks/` path. The `<repo-hooks>` line says to look in the project `tools/repohooks/` path. The `<repo-hooks>` line says to look in the project
with the name `platform/tools/repohooks` for hooks to run during the with the name `platform/tools/repohooks` for hooks to run during the

View File

@ -1,8 +1,5 @@
# Microsoft Windows Details # Microsoft Windows Details
> **Warning: The "master" branch is no longer used. Use "main" instead.**<br>
> https://gerrit.googlesource.com/git-repo/+/HEAD/docs/windows.md
Repo is primarily developed on Linux with a lot of users on macOS. Repo is primarily developed on Linux with a lot of users on macOS.
Windows is, unfortunately, not a common platform. Windows is, unfortunately, not a common platform.
There is support in repo for Windows, but there might be some rough edges. There is support in repo for Windows, but there might be some rough edges.

139
hooks.py
View File

@ -14,9 +14,11 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
import errno
import json import json
import os import os
import re import re
import subprocess
import sys import sys
import traceback import traceback
@ -33,6 +35,7 @@ else:
urllib.parse = urlparse urllib.parse = urlparse
input = raw_input # noqa: F821 input = raw_input # noqa: F821
class RepoHook(object): class RepoHook(object):
"""A RepoHook contains information about a script to run as a hook. """A RepoHook contains information about a script to run as a hook.
@ -45,13 +48,29 @@ class RepoHook(object):
Hooks are always python. When a hook is run, we will load the hook into the Hooks are always python. When a hook is run, we will load the hook into the
interpreter and execute its main() function. interpreter and execute its main() function.
Combinations of hook option flags:
- no-verify=False, verify=False (DEFAULT):
If stdout is a tty, can prompt about running hooks if needed.
If user denies running hooks, the action is cancelled. If stdout is
not a tty and we would need to prompt about hooks, action is
cancelled.
- no-verify=False, verify=True:
Always run hooks with no prompt.
- no-verify=True, verify=False:
Never run hooks, but run action anyway (AKA bypass hooks).
- no-verify=True, verify=True:
Invalid
""" """
def __init__(self, def __init__(self,
hook_type, hook_type,
hooks_project, hooks_project,
topdir, repo_topdir,
manifest_url, manifest_url,
bypass_hooks=False,
allow_all_hooks=False,
ignore_hooks=False,
abort_if_user_denies=False): abort_if_user_denies=False):
"""RepoHook constructor. """RepoHook constructor.
@ -59,20 +78,27 @@ class RepoHook(object):
hook_type: A string representing the type of hook. This is also used hook_type: A string representing the type of hook. This is also used
to figure out the name of the file containing the hook. For to figure out the name of the file containing the hook. For
example: 'pre-upload'. example: 'pre-upload'.
hooks_project: The project containing the repo hooks. If you have a hooks_project: The project containing the repo hooks.
manifest, this is manifest.repo_hooks_project. OK if this is None, If you have a manifest, this is manifest.repo_hooks_project.
which will make the hook a no-op. OK if this is None, which will make the hook a no-op.
topdir: Repo's top directory (the one containing the .repo directory). repo_topdir: The top directory of the repo client checkout.
Scripts will run with CWD as this directory. If you have a manifest, This is the one containing the .repo directory. Scripts will
this is manifest.topdir run with CWD as this directory.
If you have a manifest, this is manifest.topdir.
manifest_url: The URL to the manifest git repo. manifest_url: The URL to the manifest git repo.
abort_if_user_denies: If True, we'll throw a HookError() if the user bypass_hooks: If True, then 'Do not run the hook'.
allow_all_hooks: If True, then 'Run the hook without prompting'.
ignore_hooks: If True, then 'Do not abort action if hooks fail'.
abort_if_user_denies: If True, we'll abort running the hook if the user
doesn't allow us to run the hook. doesn't allow us to run the hook.
""" """
self._hook_type = hook_type self._hook_type = hook_type
self._hooks_project = hooks_project self._hooks_project = hooks_project
self._repo_topdir = repo_topdir
self._manifest_url = manifest_url self._manifest_url = manifest_url
self._topdir = topdir self._bypass_hooks = bypass_hooks
self._allow_all_hooks = allow_all_hooks
self._ignore_hooks = ignore_hooks
self._abort_if_user_denies = abort_if_user_denies self._abort_if_user_denies = abort_if_user_denies
# Store the full path to the script for convenience. # Store the full path to the script for convenience.
@ -108,7 +134,7 @@ class RepoHook(object):
# NOTE: Local (non-committed) changes will not be factored into this hash. # NOTE: Local (non-committed) changes will not be factored into this hash.
# I think this is OK, since we're really only worried about warning the user # I think this is OK, since we're really only worried about warning the user
# about upstream changes. # about upstream changes.
return self._hooks_project.work_git.rev_parse('HEAD') return self._hooks_project.work_git.rev_parse(HEAD)
def _GetMustVerb(self): def _GetMustVerb(self):
"""Return 'must' if the hook is required; 'should' if not.""" """Return 'must' if the hook is required; 'should' if not."""
@ -347,7 +373,7 @@ context['main'](**kwargs)
try: try:
# Always run hooks with CWD as topdir. # Always run hooks with CWD as topdir.
os.chdir(self._topdir) os.chdir(self._repo_topdir)
# Put the hook dir as the first item of sys.path so hooks can do # Put the hook dir as the first item of sys.path so hooks can do
# relative imports. We want to replace the repo dir as [0] so # relative imports. We want to replace the repo dir as [0] so
@ -397,7 +423,12 @@ context['main'](**kwargs)
sys.path = orig_syspath sys.path = orig_syspath
os.chdir(orig_path) os.chdir(orig_path)
def Run(self, user_allows_all_hooks, **kwargs): def _CheckHook(self):
# Bail with a nice error if we can't find the hook.
if not os.path.isfile(self._script_fullpath):
raise HookError('Couldn\'t find repo hook: %s' % self._script_fullpath)
def Run(self, **kwargs):
"""Run the hook. """Run the hook.
If the hook doesn't exist (because there is no hooks project or because If the hook doesn't exist (because there is no hooks project or because
@ -410,22 +441,80 @@ context['main'](**kwargs)
to the hook type. For instance, pre-upload hooks will contain to the hook type. For instance, pre-upload hooks will contain
a project_list. a project_list.
Raises: Returns:
HookError: If there was a problem finding the hook or the user declined True: On success or ignore hooks by user-request
to run a required hook (from _CheckForHookApproval). False: The hook failed. The caller should respond with aborting the action.
Some examples in which False is returned:
* Finding the hook failed while it was enabled, or
* the user declined to run a required hook (from _CheckForHookApproval)
In all these cases the user did not pass the proper arguments to
ignore the result through the option combinations as listed in
AddHookOptionGroup().
""" """
# No-op if there is no hooks project or if hook is disabled. # Do not do anything in case bypass_hooks is set, or
if ((not self._hooks_project) or (self._hook_type not in # no-op if there is no hooks project or if hook is disabled.
self._hooks_project.enabled_repo_hooks)): if (self._bypass_hooks or
return not self._hooks_project or
self._hook_type not in self._hooks_project.enabled_repo_hooks):
return True
# Bail with a nice error if we can't find the hook. passed = True
if not os.path.isfile(self._script_fullpath): try:
raise HookError('Couldn\'t find repo hook: "%s"' % self._script_fullpath) self._CheckHook()
# Make sure the user is OK with running the hook. # Make sure the user is OK with running the hook.
if (not user_allows_all_hooks) and (not self._CheckForHookApproval()): if self._allow_all_hooks or self._CheckForHookApproval():
return
# Run the hook with the same version of python we're using. # Run the hook with the same version of python we're using.
self._ExecuteHook(**kwargs) self._ExecuteHook(**kwargs)
except SystemExit as e:
passed = False
print('ERROR: %s hooks exited with exit code: %s' % (self._hook_type, str(e)),
file=sys.stderr)
except HookError as e:
passed = False
print('ERROR: %s' % str(e), file=sys.stderr)
if not passed and self._ignore_hooks:
print('\nWARNING: %s hooks failed, but continuing anyways.' % self._hook_type,
file=sys.stderr)
passed = True
return passed
@classmethod
def FromSubcmd(cls, manifest, opt, *args, **kwargs):
"""Method to construct the repo hook class
Args:
manifest: The current active manifest for this command from which we
extract a couple of fields.
opt: Contains the commandline options for the action of this hook.
It should contain the options added by AddHookOptionGroup() in which
we are interested in RepoHook execution.
"""
for key in ('bypass_hooks', 'allow_all_hooks', 'ignore_hooks'):
kwargs.setdefault(key, getattr(opt, key))
kwargs.update({
'hooks_project': manifest.repo_hooks_project,
'repo_topdir': manifest.topdir,
'manifest_url': manifest.manifestProject.GetRemote('origin').url,
})
return cls(*args, **kwargs)
@staticmethod
def AddOptionGroup(parser, name):
"""Help options relating to the various hooks."""
# Note that verify and no-verify are NOT opposites of each other, which
# is why they store to different locations. We are using them to match
# 'git commit' syntax.
group = parser.add_option_group(name + ' hooks')
group.add_option('--no-verify',
dest='bypass_hooks', action='store_true',
help='Do not run the %s hook.' % name)
group.add_option('--verify',
dest='allow_all_hooks', action='store_true',
help='Run the %s hook without prompting.' % name)
group.add_option('--ignore-hooks',
action='store_true',
help='Do not abort if %s hooks fail.' % name)

18
main.py
View File

@ -1,4 +1,4 @@
#!/usr/bin/env python #!/usr/bin/env python3
# -*- coding:utf-8 -*- # -*- coding:utf-8 -*-
# #
# Copyright (C) 2008 The Android Open Source Project # Copyright (C) 2008 The Android Open Source Project
@ -63,7 +63,7 @@ from error import NoManifestException
from error import NoSuchProjectError from error import NoSuchProjectError
from error import RepoChangedException from error import RepoChangedException
import gitc_utils import gitc_utils
from manifest_xml import GitcManifest, XmlManifest from manifest_xml import GitcClient, RepoClient
from pager import RunPager, TerminatePager from pager import RunPager, TerminatePager
from wrapper import WrapperPath, Wrapper from wrapper import WrapperPath, Wrapper
@ -85,9 +85,10 @@ MIN_PYTHON_VERSION_SOFT = (3, 6)
MIN_PYTHON_VERSION_HARD = (3, 4) MIN_PYTHON_VERSION_HARD = (3, 4)
if sys.version_info.major < 3: if sys.version_info.major < 3:
print('repo: warning: Python 2 is no longer supported; ' print('repo: error: Python 2 is no longer supported; '
'Please upgrade to Python {}.{}+.'.format(*MIN_PYTHON_VERSION_SOFT), 'Please upgrade to Python {}.{}+.'.format(*MIN_PYTHON_VERSION_SOFT),
file=sys.stderr) file=sys.stderr)
sys.exit(1)
else: else:
if sys.version_info < MIN_PYTHON_VERSION_HARD: if sys.version_info < MIN_PYTHON_VERSION_HARD:
print('repo: error: Python 3 version is too old; ' print('repo: error: Python 3 version is too old; '
@ -211,14 +212,15 @@ class _Repo(object):
return 1 return 1
cmd.repodir = self.repodir cmd.repodir = self.repodir
cmd.manifest = XmlManifest(cmd.repodir) cmd.client = RepoClient(cmd.repodir)
cmd.manifest = cmd.client.manifest
cmd.gitc_manifest = None cmd.gitc_manifest = None
gitc_client_name = gitc_utils.parse_clientdir(os.getcwd()) gitc_client_name = gitc_utils.parse_clientdir(os.getcwd())
if gitc_client_name: if gitc_client_name:
cmd.gitc_manifest = GitcManifest(cmd.repodir, gitc_client_name) cmd.gitc_manifest = GitcClient(cmd.repodir, gitc_client_name)
cmd.manifest.isGitcClient = True cmd.client.isGitcClient = True
Editor.globalConfig = cmd.manifest.globalConfig Editor.globalConfig = cmd.client.globalConfig
if not isinstance(cmd, MirrorSafeCommand) and cmd.manifest.IsMirror: if not isinstance(cmd, MirrorSafeCommand) and cmd.manifest.IsMirror:
print("fatal: '%s' requires a working directory" % name, print("fatal: '%s' requires a working directory" % name,
@ -246,7 +248,7 @@ class _Repo(object):
return 1 return 1
if gopts.pager is not False and not isinstance(cmd, InteractiveCommand): if gopts.pager is not False and not isinstance(cmd, InteractiveCommand):
config = cmd.manifest.globalConfig config = cmd.client.globalConfig
if gopts.pager: if gopts.pager:
use_pager = True use_pager = True
else: else:

View File

@ -187,12 +187,24 @@ class _XmlRemote(object):
class XmlManifest(object): class XmlManifest(object):
"""manages the repo configuration file""" """manages the repo configuration file"""
def __init__(self, repodir): def __init__(self, repodir, manifest_file, local_manifests=None):
"""Initialize.
Args:
repodir: Path to the .repo/ dir for holding all internal checkout state.
It must be in the top directory of the repo client checkout.
manifest_file: Full path to the manifest file to parse. This will usually
be |repodir|/|MANIFEST_FILE_NAME|.
local_manifests: Full path to the directory of local override manifests.
This will usually be |repodir|/|LOCAL_MANIFESTS_DIR_NAME|.
"""
# TODO(vapier): Move this out of this class.
self.globalConfig = GitConfig.ForUser()
self.repodir = os.path.abspath(repodir) self.repodir = os.path.abspath(repodir)
self.topdir = os.path.dirname(self.repodir) self.topdir = os.path.dirname(self.repodir)
self.manifestFile = os.path.join(self.repodir, MANIFEST_FILE_NAME) self.manifestFile = manifest_file
self.globalConfig = GitConfig.ForUser() self.local_manifests = local_manifests
self.isGitcClient = False
self._load_local_manifests = True self._load_local_manifests = True
self.repoProject = MetaProject(self, 'repo', self.repoProject = MetaProject(self, 'repo',
@ -283,9 +295,8 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
def _ParseGroups(self, groups): def _ParseGroups(self, groups):
return [x for x in re.split(r'[,\s]+', groups) if x] return [x for x in re.split(r'[,\s]+', groups) if x]
def Save(self, fd, peg_rev=False, peg_rev_upstream=True, peg_rev_dest_branch=True, groups=None): def ToXml(self, peg_rev=False, peg_rev_upstream=True, peg_rev_dest_branch=True, groups=None):
"""Write the current manifest out to the given file descriptor. """Return the current manifest XML."""
"""
mp = self.manifestProject mp = self.manifestProject
if groups is None: if groups is None:
@ -459,6 +470,56 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
' '.join(self._repo_hooks_project.enabled_repo_hooks)) ' '.join(self._repo_hooks_project.enabled_repo_hooks))
root.appendChild(e) root.appendChild(e)
return doc
def ToDict(self, **kwargs):
"""Return the current manifest as a dictionary."""
# Elements that may only appear once.
SINGLE_ELEMENTS = {
'notice',
'default',
'manifest-server',
'repo-hooks',
}
# Elements that may be repeated.
MULTI_ELEMENTS = {
'remote',
'remove-project',
'project',
'extend-project',
'include',
# These are children of 'project' nodes.
'annotation',
'project',
'copyfile',
'linkfile',
}
doc = self.ToXml(**kwargs)
ret = {}
def append_children(ret, node):
for child in node.childNodes:
if child.nodeType == xml.dom.Node.ELEMENT_NODE:
attrs = child.attributes
element = dict((attrs.item(i).localName, attrs.item(i).value)
for i in range(attrs.length))
if child.nodeName in SINGLE_ELEMENTS:
ret[child.nodeName] = element
elif child.nodeName in MULTI_ELEMENTS:
ret.setdefault(child.nodeName, []).append(element)
else:
raise ManifestParseError('Unhandled element "%s"' % (child.nodeName,))
append_children(element, child)
append_children(ret, doc.firstChild)
return ret
def Save(self, fd, **kwargs):
"""Write the current manifest out to the given file descriptor."""
doc = self.ToXml(**kwargs)
doc.writexml(fd, '', ' ', '\n', 'UTF-8') doc.writexml(fd, '', ' ', '\n', 'UTF-8')
def _output_manifest_project_extras(self, p, e): def _output_manifest_project_extras(self, p, e):
@ -553,20 +614,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
nodes.append(self._ParseManifestXml(self.manifestFile, nodes.append(self._ParseManifestXml(self.manifestFile,
self.manifestProject.worktree)) self.manifestProject.worktree))
if self._load_local_manifests: if self._load_local_manifests and self.local_manifests:
if os.path.exists(os.path.join(self.repodir, LOCAL_MANIFEST_NAME)):
print('error: %s is not supported; put local manifests in `%s`'
'instead' % (LOCAL_MANIFEST_NAME,
os.path.join(self.repodir, LOCAL_MANIFESTS_DIR_NAME)),
file=sys.stderr)
sys.exit(1)
local_dir = os.path.abspath(os.path.join(self.repodir,
LOCAL_MANIFESTS_DIR_NAME))
try: try:
for local_file in sorted(platform_utils.listdir(local_dir)): for local_file in sorted(platform_utils.listdir(self.local_manifests)):
if local_file.endswith('.xml'): if local_file.endswith('.xml'):
local = os.path.join(local_dir, local_file) local = os.path.join(self.local_manifests, local_file)
nodes.append(self._ParseManifestXml(local, self.repodir)) nodes.append(self._ParseManifestXml(local, self.repodir))
except OSError: except OSError:
pass pass
@ -585,7 +637,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
self._loaded = True self._loaded = True
def _ParseManifestXml(self, path, include_root): def _ParseManifestXml(self, path, include_root, parent_groups=''):
try: try:
root = xml.dom.minidom.parse(path) root = xml.dom.minidom.parse(path)
except (OSError, xml.parsers.expat.ExpatError) as e: except (OSError, xml.parsers.expat.ExpatError) as e:
@ -604,12 +656,17 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
for node in manifest.childNodes: for node in manifest.childNodes:
if node.nodeName == 'include': if node.nodeName == 'include':
name = self._reqatt(node, 'name') name = self._reqatt(node, 'name')
include_groups = ''
if parent_groups:
include_groups = parent_groups
if node.hasAttribute('groups'):
include_groups = node.getAttribute('groups') + ',' + include_groups
fp = os.path.join(include_root, name) fp = os.path.join(include_root, name)
if not os.path.isfile(fp): if not os.path.isfile(fp):
raise ManifestParseError("include %s doesn't exist or isn't a file" raise ManifestParseError("include %s doesn't exist or isn't a file"
% (name,)) % (name,))
try: try:
nodes.extend(self._ParseManifestXml(fp, include_root)) nodes.extend(self._ParseManifestXml(fp, include_root, include_groups))
# should isolate this to the exact exception, but that's # should isolate this to the exact exception, but that's
# tricky. actual parsing implementation may vary. # tricky. actual parsing implementation may vary.
except (KeyboardInterrupt, RuntimeError, SystemExit): except (KeyboardInterrupt, RuntimeError, SystemExit):
@ -618,6 +675,11 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
raise ManifestParseError( raise ManifestParseError(
"failed parsing included manifest %s: %s" % (name, e)) "failed parsing included manifest %s: %s" % (name, e))
else: else:
if parent_groups and node.nodeName == 'project':
nodeGroups = parent_groups
if node.hasAttribute('groups'):
nodeGroups = node.getAttribute('groups') + ',' + nodeGroups
node.setAttribute('groups', nodeGroups)
nodes.append(node) nodes.append(node)
return nodes return nodes
@ -1204,15 +1266,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
class GitcManifest(XmlManifest): class GitcManifest(XmlManifest):
"""Parser for GitC (git-in-the-cloud) manifests."""
def __init__(self, repodir, gitc_client_name):
"""Initialize the GitcManifest object."""
super(GitcManifest, self).__init__(repodir)
self.isGitcClient = True
self.gitc_client_name = gitc_client_name
self.gitc_client_dir = os.path.join(gitc_utils.get_gitc_manifest_dir(),
gitc_client_name)
self.manifestFile = os.path.join(self.gitc_client_dir, '.manifest')
def _ParseProject(self, node, parent=None): def _ParseProject(self, node, parent=None):
"""Override _ParseProject and add support for GITC specific attributes.""" """Override _ParseProject and add support for GITC specific attributes."""
@ -1223,3 +1277,38 @@ class GitcManifest(XmlManifest):
"""Output GITC Specific Project attributes""" """Output GITC Specific Project attributes"""
if p.old_revision: if p.old_revision:
e.setAttribute('old-revision', str(p.old_revision)) e.setAttribute('old-revision', str(p.old_revision))
class RepoClient(XmlManifest):
"""Manages a repo client checkout."""
def __init__(self, repodir, manifest_file=None):
self.isGitcClient = False
if os.path.exists(os.path.join(repodir, LOCAL_MANIFEST_NAME)):
print('error: %s is not supported; put local manifests in `%s` instead' %
(LOCAL_MANIFEST_NAME, os.path.join(repodir, LOCAL_MANIFESTS_DIR_NAME)),
file=sys.stderr)
sys.exit(1)
if manifest_file is None:
manifest_file = os.path.join(repodir, MANIFEST_FILE_NAME)
local_manifests = os.path.abspath(os.path.join(repodir, LOCAL_MANIFESTS_DIR_NAME))
super(RepoClient, self).__init__(repodir, manifest_file, local_manifests)
# TODO: Completely separate manifest logic out of the client.
self.manifest = self
class GitcClient(RepoClient, GitcManifest):
"""Manages a GitC client checkout."""
def __init__(self, repodir, gitc_client_name):
"""Initialize the GitcManifest object."""
self.gitc_client_name = gitc_client_name
self.gitc_client_dir = os.path.join(gitc_utils.get_gitc_manifest_dir(),
gitc_client_name)
super(GitcManifest, self).__init__(
repodir, os.path.join(self.gitc_client_dir, '.manifest'))
self.isGitcClient = True

View File

@ -62,7 +62,8 @@ RETRY_JITTER_PERCENT = 0.1
def _lwrite(path, content): def _lwrite(path, content):
lock = '%s.lock' % path lock = '%s.lock' % path
with open(lock, 'w') as fd: # Maintain Unix line endings on all OS's to match git behavior.
with open(lock, 'w', newline='\n') as fd:
fd.write(content) fd.write(content)
try: try:
@ -510,7 +511,7 @@ class Project(object):
with exponential backoff and jitter. with exponential backoff and jitter.
old_revision: saved git commit id for open GITC projects. old_revision: saved git commit id for open GITC projects.
""" """
self.manifest = manifest self.client = self.manifest = manifest
self.name = name self.name = name
self.remote = remote self.remote = remote
self.gitdir = gitdir.replace('\\', '/') self.gitdir = gitdir.replace('\\', '/')
@ -551,7 +552,7 @@ class Project(object):
self.linkfiles = [] self.linkfiles = []
self.annotations = [] self.annotations = []
self.config = GitConfig.ForRepository(gitdir=self.gitdir, self.config = GitConfig.ForRepository(gitdir=self.gitdir,
defaults=self.manifest.globalConfig) defaults=self.client.globalConfig)
if self.worktree: if self.worktree:
self.work_git = self._GitGetByExec(self, bare=False, gitdir=gitdir) self.work_git = self._GitGetByExec(self, bare=False, gitdir=gitdir)
@ -1026,6 +1027,7 @@ class Project(object):
if GitCommand(self, cmd, bare=True).Wait() != 0: if GitCommand(self, cmd, bare=True).Wait() != 0:
raise UploadError('Upload failed') raise UploadError('Upload failed')
if not dryrun:
msg = "posted to %s for %s" % (branch.remote.review, dest_branch) msg = "posted to %s for %s" % (branch.remote.review, dest_branch)
self.bare_git.UpdateRef(R_PUB + branch.name, self.bare_git.UpdateRef(R_PUB + branch.name,
R_HEADS + branch.name, R_HEADS + branch.name,
@ -1168,7 +1170,7 @@ class Project(object):
self._InitHooks() self._InitHooks()
def _CopyAndLinkFiles(self): def _CopyAndLinkFiles(self):
if self.manifest.isGitcClient: if self.client.isGitcClient:
return return
for copyfile in self.copyfiles: for copyfile in self.copyfiles:
copyfile._Copy() copyfile._Copy()
@ -2701,12 +2703,14 @@ class Project(object):
# Some platforms (e.g. Windows) won't let us update dotgit in situ because # Some platforms (e.g. Windows) won't let us update dotgit in situ because
# of file permissions. Delete it and recreate it from scratch to avoid. # of file permissions. Delete it and recreate it from scratch to avoid.
platform_utils.remove(dotgit) platform_utils.remove(dotgit)
# Use relative path from checkout->worktree. # Use relative path from checkout->worktree & maintain Unix line endings
with open(dotgit, 'w') as fp: # on all OS's to match git behavior.
with open(dotgit, 'w', newline='\n') as fp:
print('gitdir:', os.path.relpath(git_worktree_path, self.worktree), print('gitdir:', os.path.relpath(git_worktree_path, self.worktree),
file=fp) file=fp)
# Use relative path from worktree->checkout. # Use relative path from worktree->checkout & maintain Unix line endings
with open(os.path.join(git_worktree_path, 'gitdir'), 'w') as fp: # on all OS's to match git behavior.
with open(os.path.join(git_worktree_path, 'gitdir'), 'w', newline='\n') as fp:
print(os.path.relpath(dotgit, git_worktree_path), file=fp) print(os.path.relpath(dotgit, git_worktree_path), file=fp)
self._InitMRef() self._InitMRef()

View File

@ -1,5 +1,2 @@
> **Warning: The "master" branch is no longer used. Use "main" instead.**<br>
> https://gerrit.googlesource.com/git-repo/+/HEAD/release/README.md
These are helper tools for managing official releases. These are helper tools for managing official releases.
See the [release process](../docs/release-process.md) document for more details. See the [release process](../docs/release-process.md) document for more details.

View File

@ -16,7 +16,7 @@
from color import Coloring from color import Coloring
from command import PagedCommand from command import PagedCommand
from manifest_xml import XmlManifest from manifest_xml import RepoClient
class _Coloring(Coloring): class _Coloring(Coloring):
@ -183,7 +183,7 @@ synced and their revisions won't be found.
self.OptionParser.error('missing manifests to diff') self.OptionParser.error('missing manifests to diff')
def Execute(self, opt, args): def Execute(self, opt, args):
self.out = _Coloring(self.manifest.globalConfig) self.out = _Coloring(self.client.globalConfig)
self.printText = self.out.nofmt_printer('text') self.printText = self.out.nofmt_printer('text')
if opt.color: if opt.color:
self.printProject = self.out.nofmt_printer('project', attr='bold') self.printProject = self.out.nofmt_printer('project', attr='bold')
@ -193,12 +193,12 @@ synced and their revisions won't be found.
else: else:
self.printProject = self.printAdded = self.printRemoved = self.printRevision = self.printText self.printProject = self.printAdded = self.printRemoved = self.printRevision = self.printText
manifest1 = XmlManifest(self.manifest.repodir) manifest1 = RepoClient(self.manifest.repodir)
manifest1.Override(args[0], load_local_manifests=False) manifest1.Override(args[0], load_local_manifests=False)
if len(args) == 1: if len(args) == 1:
manifest2 = self.manifest manifest2 = self.manifest
else: else:
manifest2 = XmlManifest(self.manifest.repodir) manifest2 = RepoClient(self.manifest.repodir)
manifest2.Override(args[1], load_local_manifests=False) manifest2.Override(args[1], load_local_manifests=False)
diff = manifest1.projectsDiff(manifest2) diff = manifest1.projectsDiff(manifest2)

View File

@ -65,7 +65,7 @@ Displays detailed usage information about a command.
def gitc_supported(cmd): def gitc_supported(cmd):
if not isinstance(cmd, GitcAvailableCommand) and not isinstance(cmd, GitcClientCommand): if not isinstance(cmd, GitcAvailableCommand) and not isinstance(cmd, GitcClientCommand):
return True return True
if self.manifest.isGitcClient: if self.client.isGitcClient:
return True return True
if isinstance(cmd, GitcClientCommand): if isinstance(cmd, GitcClientCommand):
return False return False
@ -127,7 +127,7 @@ Displays detailed usage information about a command.
self.wrap.end_paragraph(1) self.wrap.end_paragraph(1)
self.wrap.end_paragraph(0) self.wrap.end_paragraph(0)
out = _Out(self.manifest.globalConfig) out = _Out(self.client.globalConfig)
out._PrintSection('Summary', 'helpSummary') out._PrintSection('Summary', 'helpSummary')
cmd.OptionParser.print_help() cmd.OptionParser.print_help()
out._PrintSection('Description', 'helpDescription') out._PrintSection('Description', 'helpDescription')

View File

@ -44,7 +44,7 @@ class Info(PagedCommand):
help="Disable all remote operations") help="Disable all remote operations")
def Execute(self, opt, args): def Execute(self, opt, args):
self.out = _Coloring(self.manifest.globalConfig) self.out = _Coloring(self.client.globalConfig)
self.heading = self.out.printer('heading', attr='bold') self.heading = self.out.printer('heading', attr='bold')
self.headtext = self.out.nofmt_printer('headtext', fg='yellow') self.headtext = self.out.nofmt_printer('headtext', fg='yellow')
self.redtext = self.out.printer('redtext', fg='red') self.redtext = self.out.printer('redtext', fg='red')

View File

@ -365,7 +365,7 @@ to update the working directory files.
return a return a
def _ShouldConfigureUser(self, opt): def _ShouldConfigureUser(self, opt):
gc = self.manifest.globalConfig gc = self.client.globalConfig
mp = self.manifest.manifestProject mp = self.manifest.manifestProject
# If we don't have local settings, get from global. # If we don't have local settings, get from global.
@ -414,7 +414,7 @@ to update the working directory files.
return False return False
def _ConfigureColor(self): def _ConfigureColor(self):
gc = self.manifest.globalConfig gc = self.client.globalConfig
if self._HasColorSet(gc): if self._HasColorSet(gc):
return return

View File

@ -15,6 +15,8 @@
# limitations under the License. # limitations under the License.
from __future__ import print_function from __future__ import print_function
import json
import os import os
import sys import sys
@ -68,6 +70,10 @@ to indicate the remote ref to push changes to via 'repo upload'.
help='If in -r mode, do not write the dest-branch field. ' help='If in -r mode, do not write the dest-branch field. '
'Only of use if the branch names for a sha1 manifest are ' 'Only of use if the branch names for a sha1 manifest are '
'sensitive.') 'sensitive.')
p.add_option('--json', default=False, action='store_true',
help='Output manifest in JSON format (experimental).')
p.add_option('--pretty', default=False, action='store_true',
help='Format output for humans to read.')
p.add_option('-o', '--output-file', p.add_option('-o', '--output-file',
dest='output_file', dest='output_file',
default='-', default='-',
@ -83,6 +89,22 @@ to indicate the remote ref to push changes to via 'repo upload'.
fd = sys.stdout fd = sys.stdout
else: else:
fd = open(opt.output_file, 'w') fd = open(opt.output_file, 'w')
if opt.json:
print('warning: --json is experimental!', file=sys.stderr)
doc = self.manifest.ToDict(peg_rev=opt.peg_rev,
peg_rev_upstream=opt.peg_rev_upstream,
peg_rev_dest_branch=opt.peg_rev_dest_branch)
json_settings = {
# JSON style guide says Uunicode characters are fully allowed.
'ensure_ascii': False,
# We use 2 space indent to match JSON style guide.
'indent': 2 if opt.pretty else None,
'separators': (',', ': ') if opt.pretty else (',', ':'),
'sort_keys': True,
}
fd.write(json.dumps(doc, **json_settings))
else:
self.manifest.Save(fd, self.manifest.Save(fd,
peg_rev=opt.peg_rev, peg_rev=opt.peg_rev,
peg_rev_upstream=opt.peg_rev_upstream, peg_rev_upstream=opt.peg_rev_upstream,

View File

@ -165,7 +165,7 @@ the following meanings:
proj_dirs, proj_dirs_parents, outstring) proj_dirs, proj_dirs_parents, outstring)
if outstring: if outstring:
output = StatusColoring(self.manifest.globalConfig) output = StatusColoring(self.client.globalConfig)
output.project('Objects not within a project (orphans)') output.project('Objects not within a project (orphans)')
output.nl() output.nl()
for entry in outstring: for entry in outstring:

View File

@ -780,6 +780,7 @@ later is required to fix a server side protocol bug.
start = time.time() start = time.time()
success = mp.Sync_NetworkHalf(quiet=opt.quiet, verbose=opt.verbose, success = mp.Sync_NetworkHalf(quiet=opt.quiet, verbose=opt.verbose,
current_branch_only=opt.current_branch_only, current_branch_only=opt.current_branch_only,
force_sync=opt.force_sync,
tags=opt.tags, tags=opt.tags,
optimized_fetch=opt.optimized_fetch, optimized_fetch=opt.optimized_fetch,
retry_fetches=opt.retry_fetches, retry_fetches=opt.retry_fetches,

View File

@ -21,7 +21,7 @@ import sys
from command import InteractiveCommand from command import InteractiveCommand
from editor import Editor from editor import Editor
from error import HookError, UploadError from error import UploadError
from git_command import GitCommand from git_command import GitCommand
from git_refs import R_HEADS from git_refs import R_HEADS
from hooks import RepoHook from hooks import RepoHook
@ -205,33 +205,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
p.add_option('--no-cert-checks', p.add_option('--no-cert-checks',
dest='validate_certs', action='store_false', default=True, dest='validate_certs', action='store_false', default=True,
help='Disable verifying ssl certs (unsafe).') help='Disable verifying ssl certs (unsafe).')
RepoHook.AddOptionGroup(p, 'pre-upload')
# Options relating to upload hook. Note that verify and no-verify are NOT
# opposites of each other, which is why they store to different locations.
# We are using them to match 'git commit' syntax.
#
# Combinations:
# - no-verify=False, verify=False (DEFAULT):
# If stdout is a tty, can prompt about running upload hooks if needed.
# If user denies running hooks, the upload is cancelled. If stdout is
# not a tty and we would need to prompt about upload hooks, upload is
# cancelled.
# - no-verify=False, verify=True:
# Always run upload hooks with no prompt.
# - no-verify=True, verify=False:
# Never run upload hooks, but upload anyway (AKA bypass hooks).
# - no-verify=True, verify=True:
# Invalid
g = p.add_option_group('Upload hooks')
g.add_option('--no-verify',
dest='bypass_hooks', action='store_true',
help='Do not run the upload hook.')
g.add_option('--verify',
dest='allow_all_hooks', action='store_true',
help='Run the upload hook without prompting.')
g.add_option('--ignore-hooks',
dest='ignore_hooks', action='store_true',
help='Do not abort uploading if upload hooks fail.')
def _SingleBranch(self, opt, branch, people): def _SingleBranch(self, opt, branch, people):
project = branch.project project = branch.project
@ -554,10 +528,10 @@ Gerrit Code Review: https://www.gerritcodereview.com/
avail = [up_branch] avail = [up_branch]
else: else:
avail = None avail = None
print('ERROR: Current branch (%s) not uploadable. ' print('repo: error: Unable to upload branch "%s". '
'You may be able to type ' 'You might be able to fix the branch by running:\n'
'"git branch --set-upstream-to m/master" to fix ' ' git branch --set-upstream-to m/%s' %
'your branch.' % str(cbr), (str(cbr), self.manifest.branch),
file=sys.stderr) file=sys.stderr)
else: else:
avail = project.GetUploadableBranches(branch) avail = project.GetUploadableBranches(branch)
@ -572,30 +546,14 @@ Gerrit Code Review: https://www.gerritcodereview.com/
(branch,), file=sys.stderr) (branch,), file=sys.stderr)
return 1 return 1
if not opt.bypass_hooks:
hook = RepoHook('pre-upload', self.manifest.repo_hooks_project,
self.manifest.topdir,
self.manifest.manifestProject.GetRemote('origin').url,
abort_if_user_denies=True)
pending_proj_names = [project.name for (project, available) in pending] pending_proj_names = [project.name for (project, available) in pending]
pending_worktrees = [project.worktree for (project, available) in pending] pending_worktrees = [project.worktree for (project, available) in pending]
passed = True hook = RepoHook.FromSubcmd(
try: hook_type='pre-upload', manifest=self.manifest,
hook.Run(opt.allow_all_hooks, project_list=pending_proj_names, opt=opt, abort_if_user_denies=True)
worktree_list=pending_worktrees) if not hook.Run(
except SystemExit: project_list=pending_proj_names,
passed = False worktree_list=pending_worktrees):
if not opt.ignore_hooks:
raise
except HookError as e:
passed = False
print("ERROR: %s" % str(e), file=sys.stderr)
if not passed:
if opt.ignore_hooks:
print('\nWARNING: pre-upload hooks failed, but uploading anyways.',
file=sys.stderr)
else:
return 1 return 1
if opt.reviewers: if opt.reviewers:

View File

@ -19,6 +19,8 @@
from __future__ import print_function from __future__ import print_function
import os import os
import shutil
import tempfile
import unittest import unittest
import xml.dom.minidom import xml.dom.minidom
@ -146,3 +148,133 @@ class ValueTests(unittest.TestCase):
with self.assertRaises(error.ManifestParseError): with self.assertRaises(error.ManifestParseError):
node = self._get_node('<node a="xx"/>') node = self._get_node('<node a="xx"/>')
manifest_xml.XmlInt(node, 'a') manifest_xml.XmlInt(node, 'a')
class XmlManifestTests(unittest.TestCase):
"""Check manifest processing."""
def setUp(self):
self.tempdir = tempfile.mkdtemp(prefix='repo_tests')
self.repodir = os.path.join(self.tempdir, '.repo')
self.manifest_dir = os.path.join(self.repodir, 'manifests')
self.manifest_file = os.path.join(
self.repodir, manifest_xml.MANIFEST_FILE_NAME)
self.local_manifest_dir = os.path.join(
self.repodir, manifest_xml.LOCAL_MANIFESTS_DIR_NAME)
os.mkdir(self.repodir)
os.mkdir(self.manifest_dir)
# The manifest parsing really wants a git repo currently.
gitdir = os.path.join(self.repodir, 'manifests.git')
os.mkdir(gitdir)
with open(os.path.join(gitdir, 'config'), 'w') as fp:
fp.write("""[remote "origin"]
url = https://localhost:0/manifest
""")
def tearDown(self):
shutil.rmtree(self.tempdir, ignore_errors=True)
def getXmlManifest(self, data):
"""Helper to initialize a manifest for testing."""
with open(self.manifest_file, 'w') as fp:
fp.write(data)
return manifest_xml.XmlManifest(self.repodir, self.manifest_file)
def test_empty(self):
"""Parse an 'empty' manifest file."""
manifest = self.getXmlManifest(
'<?xml version="1.0" encoding="UTF-8"?>'
'<manifest></manifest>')
self.assertEqual(manifest.remotes, {})
self.assertEqual(manifest.projects, [])
def test_link(self):
"""Verify Link handling with new names."""
manifest = manifest_xml.XmlManifest(self.repodir, self.manifest_file)
with open(os.path.join(self.manifest_dir, 'foo.xml'), 'w') as fp:
fp.write('<manifest></manifest>')
manifest.Link('foo.xml')
with open(self.manifest_file) as fp:
self.assertIn('<include name="foo.xml" />', fp.read())
def test_toxml_empty(self):
"""Verify the ToXml() helper."""
manifest = self.getXmlManifest(
'<?xml version="1.0" encoding="UTF-8"?>'
'<manifest></manifest>')
self.assertEqual(manifest.ToXml().toxml(), '<?xml version="1.0" ?><manifest/>')
def test_todict_empty(self):
"""Verify the ToDict() helper."""
manifest = self.getXmlManifest(
'<?xml version="1.0" encoding="UTF-8"?>'
'<manifest></manifest>')
self.assertEqual(manifest.ToDict(), {})
def test_project_group(self):
"""Check project group settings."""
manifest = self.getXmlManifest("""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/main" />
<project name="test-name" path="test-path"/>
<project name="extras" path="path" groups="g1,g2,g1"/>
</manifest>
""")
self.assertEqual(len(manifest.projects), 2)
# Ordering isn't guaranteed.
result = {
manifest.projects[0].name: manifest.projects[0].groups,
manifest.projects[1].name: manifest.projects[1].groups,
}
project = manifest.projects[0]
self.assertCountEqual(
result['test-name'],
['name:test-name', 'all', 'path:test-path'])
self.assertCountEqual(
result['extras'],
['g1', 'g2', 'g1', 'name:extras', 'all', 'path:path'])
def test_include_levels(self):
root_m = os.path.join(self.manifest_dir, 'root.xml')
with open(root_m, 'w') as fp:
fp.write("""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/main" />
<include name="level1.xml" groups="level1-group" />
<project name="root-name1" path="root-path1" />
<project name="root-name2" path="root-path2" groups="r2g1,r2g2" />
</manifest>
""")
with open(os.path.join(self.manifest_dir, 'level1.xml'), 'w') as fp:
fp.write("""
<manifest>
<include name="level2.xml" groups="level2-group" />
<project name="level1-name1" path="level1-path1" />
</manifest>
""")
with open(os.path.join(self.manifest_dir, 'level2.xml'), 'w') as fp:
fp.write("""
<manifest>
<project name="level2-name1" path="level2-path1" groups="l2g1,l2g2" />
</manifest>
""")
include_m = manifest_xml.XmlManifest(self.repodir, root_m)
for proj in include_m.projects:
if proj.name == 'root-name1':
# Check include group not set on root level proj.
self.assertNotIn('level1-group', proj.groups)
if proj.name == 'root-name2':
# Check root proj group not removed.
self.assertIn('r2g1', proj.groups)
if proj.name == 'level1-name1':
# Check level1 proj has inherited group level 1.
self.assertIn('level1-group', proj.groups)
if proj.name == 'level2-name1':
# Check level2 proj has inherited group levels 1 and 2.
self.assertIn('level1-group', proj.groups)
self.assertIn('level2-group', proj.groups)
# Check level2 proj group not removed.
self.assertIn('l2g1', proj.groups)

View File

@ -77,7 +77,7 @@ class ReviewableBranchTests(unittest.TestCase):
# Start off with the normal details. # Start off with the normal details.
rb = project.ReviewableBranch( rb = project.ReviewableBranch(
fakeproj, fakeproj.config.GetBranch('work'), 'master') fakeproj, fakeproj.config.GetBranch('work'), 'main')
self.assertEqual('work', rb.name) self.assertEqual('work', rb.name)
self.assertEqual(1, len(rb.commits)) self.assertEqual(1, len(rb.commits))
self.assertIn('Del file', rb.commits[0]) self.assertIn('Del file', rb.commits[0])
@ -90,9 +90,9 @@ class ReviewableBranchTests(unittest.TestCase):
self.assertTrue(rb.date) self.assertTrue(rb.date)
# Now delete the tracking branch! # Now delete the tracking branch!
fakeproj.work_git.branch('-D', 'master') fakeproj.work_git.branch('-D', 'main')
rb = project.ReviewableBranch( rb = project.ReviewableBranch(
fakeproj, fakeproj.config.GetBranch('work'), 'master') fakeproj, fakeproj.config.GetBranch('work'), 'main')
self.assertEqual(0, len(rb.commits)) self.assertEqual(0, len(rb.commits))
self.assertFalse(rb.base_exists) self.assertFalse(rb.base_exists)
# Hard to assert anything useful about this. # Hard to assert anything useful about this.

View File

@ -402,8 +402,8 @@ class ResolveRepoRev(GitCheckoutTestCase):
self.assertEqual('refs/heads/stable', rrev) self.assertEqual('refs/heads/stable', rrev)
self.assertEqual(self.REV_LIST[1], lrev) self.assertEqual(self.REV_LIST[1], lrev)
rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, 'master') rrev, lrev = self.wrapper.resolve_repo_rev(self.GIT_DIR, 'main')
self.assertEqual('refs/heads/master', rrev) self.assertEqual('refs/heads/main', rrev)
self.assertEqual(self.REV_LIST[0], lrev) self.assertEqual(self.REV_LIST[0], lrev)
def test_tag_name(self): def test_tag_name(self):