Sometimes parsing JSON is easier than parsing XML, especially when
the XML format is limited (which ours is). Add a --json option to
the manifest command to quickly emit that form.
Bug: https://crbug.com/gerrit/11743
Change-Id: Ia2bb254a78ae2b70a851638b4545fcafe8c1a76b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280436
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Using sha1 manifest, project's revisionId is initialized
first by the manifest.
An update of a projet revision by extend-project node does
not apply to the revisionId which is therefore kept to the
initial value.
Resets revisionId value when revision is updated by an
extend-project node.
Change-Id: I873af283890cebaeaabde966f04b125642af929f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/275715
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Miguel Gaio <miguel.gaio@renault.com>
We deprecated this 8 years ago. Time to drop it to simplify the code
as it'll help with refactoring in this module to not migrate it.
Change-Id: I2deae5496d1f66a4491408fcdc95cd527062f8b6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/280798
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
If the manifest uses a trailing slash on the name attribute, repo
will construct bad internal filesystem paths which confuses tools
later on.
For example, this manifest entry:
<project name="aosp/platform/system/libhidl/" ...
will cause repo to use paths like:
.repo/project-objects/aosp/platform/system/libhidl/.git/
when it really should be using:
.repo/project-objects/aosp/platform/system/libhidl.git
Apply the normalization when we construct the local filesystem paths
as we cannot guarantee that the remote URL constructed from these
will behave the same. A server might really want:
https://example.com/aosp/platform/system/libhidl/
and would throw an error if we instead tried to fetch:
https://example.com/aosp/platform/system/libhidl
Unfortunately, any existing repo client checkouts that use such a
manifest will hit a one-time sync error as the internal git location
has changed. I'm not sure there's a way to cleanly migrate that.
Bug: https://crbug.com/1086043
Change-Id: I30bea0ffd23e478de89a035f408055e48a102658
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/268742
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@digital.ai>
For large projects, clone bundle is useful because it provided a way to
efficiently transfer a large portion of git objects through CDN, without
needing to interact with git server. However, with partial clones, the
intention is to not download most of the objects, so the use of clone
bundles would defeat the space savings normally seen with partial
clones, as they are downloaded before the first fetch.
A new option, --clone-bundle is added to override this behavior.
Add a new repo.clonebundle variable which remembers the choice if
explicitly given from command line at repo init.
Change-Id: I03638474af303a82af34579e16cd4700690b5f43
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/268452
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
When generating a revision locked manifest, we need to know what
ref to push changes to when doing 'repo upload'. This information
is lost when we lock the revision attribute to a particular commit
hash, so we need to expose it through the dest-branch attribute.
Bug: https://crbug.com/1005103
Test: manual execution
Change-Id: Ib31fd77ad8c9379759c4181dac1ea97de43eec35
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/263572
Tested-by: Sean McAllister <smcallis@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
We've been overly lenient with boolean parsing by ignoring invalid
values as "false" even if the user didn't intend that. Turn all
unknown values into warnings to avoid breaking existing manifests,
and unify the parsing logic in a helper to simplify.
We've been stricter about numbers, but still copying & pasting
inconsistent code. Add a helper for this too. For out of range
sync-j numbers (i.e. less than 1), throw a warning for now, but
mark it for future hard failures.
Change-Id: I924162b8036e6a5f1e31b6ebb24b6a26ed63712d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256457
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Trying to use the config state when the git tree hasn't yet been
created hits bad side effects. Add a check to avoid probing the
config logic during the first run. It's not clear what's going
wrong at the lower layers, but this gets us back to the behavior
before we added worktree support, so lets settle the status quo.
Bug: https://crbug.com/gerrit/12387
Change-Id: I85b56797455f3c2e249d02c18496e060be05501d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256592
Reviewed-by: Xin Li <delphij@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Changing this to a file instead of using a symlink serves two purposes:
* We can insert some comments & doc links to help users learn what this
is for, discover relevant documentation, and to discourage them from
modifying things.
* Windows requires Administrator access to use symlinks. With this
last change, Windows users can get repo client checkouts with the new
--worktree option and not need symlinks anywhere at all. Which means
they no longer need to be an Administrator in order to `repo sync`.
Change-Id: I9bc46824fd8d4b0f446ba84bd764994ca1e597e2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256313
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
We also need to check more things in the manifest/project handlers,
and use platform_utils in a few places to address Windows behavior.
Drop Python 2.7 from Windows testing as it definitely doesn't work
and we won't be fixing it.
Change-Id: I83d00ee9f1612312bb3f7147cb9535fc61268245
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256113
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
This provides initial support for using git worktrees internally
instead of our own ad-hoc symlink tree. It's been lightly tested
which is why it's not currently exposed via --help.
When people opt-in to worktrees in an existing repo client checkout,
no projects are migrated. Instead, only new projects will use the
worktree method. This allows for limited testing/opting in without
having to completely blow things away or get a second checkout.
Bug: https://crbug.com/gerrit/11486
Change-Id: Ic3ff891b30940a6ba497b406b2a387e0a8517ed8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254075
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
- E301 expected 1 blank line
- E302 expected 2 blank lines
- E303 too many blank lines
- E305 expected 2 blank lines after class or function definition
- E306 expected 1 blank line before a nested definition
Fixed automatically with autopep8:
git ls-files | grep py$ | xargs autopep8 --in-place \
--select E301,E302,E303,E305,E306
Manually fix issues in project.py caused by misuse of block comments.
Change-Id: Iee840fcaff48aae504ddac9c3e76d2acd484f6a9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254599
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
- E121 continuation line under-indented for hanging indent
- E122 continuation line missing indentation or outdented
- E125 continuation line with same indent as next logical line
- E126 continuation line over-indented for hanging indent
- E127 continuation line over-indented for visual indent
- E128 continuation line under-indented for visual indent
- E129 visually indented line with same indent as next logical line
- E131 continuation line unaligned for hanging indent
Fixed automatically with autopep8:
git ls-files | grep py$ | xargs autopep8 --in-place \
--select E121,E122,E125,E126,E127,E128,E129,E131
Change-Id: Ifd95fb8e6a1a4d6e9de187b5787d64a6326dd249
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254605
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
flake8 reports:
E713 test for membership should be 'not in'
Change-Id: I4446be67c431b7267105b53478d2ceba2af758d7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254451
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
flake8 reports:
W391 blank line at end of file
Change-Id: I5498b2de2d1268d4f1f4b9e1760f9fa93a6da4cd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254594
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
Some Android/Nest manifests are using <linkfile> with src="." to
create stable paths to specific projects. Allow that specific
use case as it seems reasonable to support.
Bug: https://crbug.com/gerrit/11218
Change-Id: I5eadec257cd58ba0f8687c590ddc250a7a414a85
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254276
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This commit supports for the 'remote' attribute in
<extend-project>. This avoids the need to perform a <remove-project>
followed by a <project> in local manifests.
Change-Id: I9f9347913337ec9d159bc264d15ce97881ae5398
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253092
Tested-by: Kyunam Jo <kyunam.jo@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reject paths in <copyfile> & <linkfile> that try to use symlinks or
non-file or non-dirs.
We don't fully validate <linkfile> when src is a glob as it's a bit
complicated -- any component in the src could be the glob. We make
sure the destination is a directory, and that any paths in that dir
are created as symlinks. So while this can be used to read any path,
it can't be abused to write to any paths.
Bug: https://crbug.com/gerrit/11218
Change-Id: I68b6d789b5ca4e43f569e75e8b293b3e13d3224b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233074
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reject paths in <copyfile> & <linkfile> that point outside of their
respective scopes. This validates paths while parsing the manifest
as this should be quick & cheap: we don't access the filesystem as
this code runs before we've synced.
Bug: https://crbug.com/gerrit/11218
Change-Id: I8e17bb91f3f5b905a9d76391b29fbab4cb77aa58
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/232932
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
A new option, --partial-clone is added to 'repo init' which tells repo
to utilize git's partial clone functionality, which reduces disk and
bandwidth usage when downloading by omitting blob downloads initially.
Different from restricting clone-depth, the user will have full access
to change history, etc., as the objects are downloaded on demand.
Change-Id: I60326744875eac16521a007bd7d5481112a98749
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/229532
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Xin Li <delphij@google.com>
This takes a single argument (the error message), not multiple
arguments that get formatted implicitly.
Change-Id: Idfbc913ea9f93820edb7e955e9e4f57618c8cd1b
There's no reason to support any other encoding in these files.
This only affects the files themselves and not streams they open.
Bug: https://crbug.com/gerrit/10418
Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
The current implementation ignores the user-specified paths to
manifests. if the "repo diffmanifests" is invoked with absolute
file paths for one or both manifests, the command fails with message:
fatal: duplicate path ... in /tmp/manifest-old.xml
Also the current implementation fails to expand the absolute path to
manifest files if "repo diffmanifests" is invoked with relative
paths, i.e "repo diffmanifests manifest-old.xml manifest-new.xml".
fatal: manifest manifest-old.xml not found
This commit fixes the first issue by disabling the local manifest
discovery for diffmanifests command, and the second issue by
expanding paths to manifests within "diffmanifests" sub-command.
Test: repo manifest --revision-as-HEAD -o /tmp/manifest-old.xml
repo sync
repo manifest --revision-as-HEAD -o /tmp/manifest-new.xml
repo diffmanifests /tmp/manifest-old.xml /tmp/manifest-new.xml
Change-Id: Ia125d769bfbea75adb9aba81abbd8c636f2168d4
Signed-off-by: Vasyl Gello <vasek.gello@gmail.com>
* Add more file i/o wrappers in platform_utils to allow using
long paths (length > MAX_PATH) on Windows.
* Paths using the long path syntax ("\\?\" prefix) should never
escape the platform_utils API surface area, so that this
specific syntax is not visible to the rest of the repo code base.
* Forward many calls from os.xxx to platform_utils.xxx in various place
to ensure long paths support, specifically when repo decides to delete
obsolete directories.
* There are more places that need to be converted to support long paths,
this commit is an initial effort to unblock a few common use cases.
* Also, fix remove function to handle directory symlinks
Change-Id: If82ccc408e516e96ff7260be25f8fd2fe3f9571a
pylint is not used since bb5b1a0. The pyflakes cleanup mentioned in that
commit has not been done, but given that this project is no longer being
actively developed I don't think it's worth spending time doing it.
Leaving the pylint suppressions causes confusion because it leads people
to think that we are still using pylint.
Change-Id: If7d9f280a0f408c780f15915ffdb80579ae21f69
It's convenient to set upstream for all projects in a manifest instead of
repeating the same value for each project.
Change-Id: I946b1de4efb01b351c332dfad108fa7d4f443cba
This change adds support for the 'revision' attribute in
<extend-project>. This avoids the need to perform a <remove-project>
followed by a <project> in local manifests.
Change-Id: Id2834fcfc1ae0d74b3347bed3618f250bf696b1f
Allow the 'default' and 'project' element in the manifest
file to apply "--no-tags" option equivalent.
Change-Id: I7e0f8c17a0e25cca744d45df049076d203c52ff5
Signed-off-by: YOUNG HO CHA <ganadist@gmail.com>
os.remove raises an exception when deleting read-only files on
Windows. Replace all calls with calls to platform_utils.remove,
which deals with deals with that issue.
Change-Id: I4dc9e0c9a36b4238880520c69f5075eca40f3e66
Replace all calls to os.symlink with platform_utils.symlink.
The Windows implementation calls into the CreateSymbolicLinkW Win32
API, as os.symlink is not supported.
Separate the Win32 API definitions into a separate module
platform_utils_win32 for clarity.
Change-Id: I0714c598664c2df93383734e609d948692c17ec5
repo sync can sync submodules via the --fetch-submodules option.
However, if the manifest repo has submodules, those will not be synced.
Having submodules in the manifest repo -- while not commonly done -- can
be useful for inheriting a manifest from another project using <include>
and layering changes on top of it. In this way, you can avoid having to
deal with merge conflicts between your own manifests and the other
project's manifests (for example, if you're managing an Android fork).
Add a --submodule option to init that automatically syncs the submodules
in the manifest repo whenever the manifest repo changes.
Change-Id: I45d34f04517774c1462d7f233f482d1d81a332a8
Signed-off-by: Martin Kelly <mkelly@xevo.com>
repo already special-cases sso:// URLs to behave similarly to https://
and rpc:// elsewhere in repo, but it forgot to do so here.
Noticed when trying to use relative URLs in a manifest obtained using
an sso:// URL.
Change-Id: Ia11469a09bbd6e444dbc4f22c82f9bbe9f5fd083
Allow the 'remote' element in the manifest file to define an optional
'pushurl' attribute which is passed into the .git/config file.
Change-Id: If342d299d371374aedc4440645798888869c9714
Signed-off-by: Steve Rae <steve.rae@raedomain.com>
When the alias attribute is set for a remote, the RemoteSpec attached to
a Project only contains the alias name used by git, not the original
name used in the manifest. But that's not enough information to
reconstruct the manifest, so save off the original manifest name as
another RemoteSpec parameter, only used to write the manifest out.
Bug: Issue 181
Bug: Issue 219
Change-Id: Id7417dfd6ce5572e4e5fe14f22924fdf088ca4f3
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.
This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.
Change-Id: I435235cb5622a048ffad0059affd32ecf71f1f5b
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.
This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.
Change-Id: I5dbc850dd73a9fbd10ab2470ae4c40e46ff894de
Updates the repo launcher and gitc_utils to pull the manifest
directory location out of the gitc config file.
Change-Id: Id08381b8a7d61962093d5cddcb3ff6afbb13004b
Add repo start support for GITC checkouts. If the user is in
the GITC FS view, they can now run repo start to check out
the sources and create a new working branch.
When "repo start" is called on a GITC project, the revision
tag is set to an empty string and saved in a new tag:
old-revision. This tells the GITC filesystem to display the
local copy of the sources when being viewed. The local copy
is created by pulling the project sources and the new branch
is created based off the original project revision.
Updated main.py to setup each command's gitc_manifest when
appropriate.
Updated repo sync's logic to sync opened projects and
updating the GITC manifest file for the rest.
Change-Id: I7e4809d1c4fc43c69b26f2f1bebe45aab0cae628
This argument wasn't being copied, which caused syncs from generated
manifests to pull down too much of the git history.
Change-Id: I269bab788d4557267c081628b3f8c6aec7744e81
Previously, in running the `manifest` command, we wouldn't output the
upstream if the default upstream would include the pinned sha1.
However, now that fetching refs/heads/* doesn't guarantee that we will
have the sha1, we need to always output the specified upstream branch.
Change-Id: Ib8b409a8ecd439397b38ee9649c530407797f841
Previously, we used a regex that would only remove a phony string from
a url if it existed, but we recently replaced that with a slice. This
change goes back to the previous behavior.
Change-Id: I8baf527be01c4b49d45b903b31a1cd6315563d5b
Instead of using regex, append the netloc and relative
scheme lists with the custom scheme.
The schemes will only be appended when needed, instead
of passing X amount of regex replaces.
see http://bugs.python.org/issue18828 for more details.
Change-Id: I10d26d5ddc32e7ed04c5a412bdd6e13ec59eb70f
This reverts commit 38e4387f8e.
A "repo init" followed by "repo sync" is meant to be as safe as
"git clone". In particular it should not run arbitrary code provided
by the manifest owner.
It would still be nice to have support for manifest-defined git hooks
--- they'd just need a prompt like the upload RepoHook has. Hopefully
a later change can bring them back.
Change-Id: I5ecd90fb5c2ed64f103d856d1ffcba38a47b062d
Signed-off-by: Jonathan Nieder <jrn@google.com>
