Compare commits

...

74 Commits

Author SHA1 Message Date
98bb76577d project: prune sample hooks
These hooks are never used and often get stale, so just trim them.
Users rarely look in these dirs to begin with.

Change-Id: Ic785aa55fb7ec84a61376df101127d0018882030
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327538
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-10 17:41:45 +00:00
d33dce0b77 project: drop support for symlinking internal .git files
Since we don't do this anymore, and there prob won't be a need to
bring it back, drop support for it.

Bug: https://crbug.com/gerrit/15460
Change-Id: I7d86706f108c797a5c7962cb1578693d49430367
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327537
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-10 17:41:40 +00:00
89ed8acdbe project: abort a bit earlier before migrating .git/
Verify all the .git/ paths will be handled by the migration logic before
starting the migration.  This way we still abort & log an error, but the
user gets to see it before we put the tree into a state that they have to
manually recover.  Also add a few more known-safe-to-clobber paths.

Bug: https://crbug.com/gerrit/15273
Change-Id: If49d69b341bc960ddcafa30da333fb5ec7145b51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327557
Reviewed-by: Colin Cross <ccross@android.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-07 20:17:14 +00:00
71e48b7672 Revert "sync: dropped "NOTICE: --use-superproject is in beta ..." message."
This reverts commit d53cb9549a. As long as
repo's reference docs treat this feature as a work in progress and don't
cover it well enough to allow all repo maintainers to easily support it,
it is inconsistent to report to users that it is no longer in beta.
Thanks for vapier@google.com for noticing.

https://crbug.com/gerrit/15527 tracks the required documentation changes
before we'd be ready to roll forward again.

Change-Id: Ic9bd951cfb3c1abf6e1bfa30dfe4afa1c9b7bec6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327337
Reviewed-by: Jonathan Nieder <jrn@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jonathan Nieder <jrn@google.com>
2022-01-06 20:01:03 +00:00
13576a8caf project: stop symlinking info dir under .git/
Unsharing this directory shouldn't be a problem.  The current repo code
treated it as a file, and while that's actually incorrect, files & dirs
are basically treated the same, so it's practically the same.

Let's enumerate each subpath since there aren't that many.

info/refs:
Only used when the project is exported over git dumb transports (i.e.
a http:// server).  Repo never does this, and it's extremely unlikely
any user has ever done this.  Plus, this proposal talks about unsharing
project refs, so this file should get unshared too.

info/grafts:
A user-configurable file that repo never touches.  Might be useful to
share across projects, but probably rarely (if ever) used by developers,
and forcing them to configure it for each project isn't that big of a
deal.

info/exclude:
info/attributes:
User-configurable files that repo never touches.  Doesn't seem like
most users ever touch these, and if they do, having them do it for
each shared project isn't a big deal.

info/sparse-checkout:
Repo doesn't use sparse checkouts, and it's extremely unlikely to even
work if a user tried doing something themselves.

Bug: https://crbug.com/gerrit/15460
Change-Id: I53e44d73a6d7a92da615b46600d8ea51cb46e3ac
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327519
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-06 08:31:45 +00:00
2345906d04 project: stop symlinking description file under .git/
Nothing uses this path.  It’s only for exporting git dirs e.g. for
online gitweb use which probably no one does.  It is not the same
description file as exists on servers we cloned from.  Leaving it
as the default plain text file will simplify code.

We don't undo any existing symlinks if they exist since repo does
not care about them, and their existence doesn't hurt.

Bug: https://crbug.com/gerrit/15460
Change-Id: Ic34fe7c3cfb8f6da844de5be30158f59382b1cc8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327518
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-06 08:29:06 +00:00
41289c62b4 project: stop symlinking svn under .git/
This path only matters to users of `git svn` who manually run it in
local projects after they get a full repo client checkout.  With svn
usage falling in general, and with the fact that the source checkout
now symlinks its .git/ state to the internal projects/ path, we don't
need to manage this anymore.

It means the path won't be shared among multiple local projects that
have the same remote, but so it goes.  It was an optimization only,
not functionality required for correctness.  We want to simplify the
internals to stop messing with git state, and this particular path
doesn't seem worth the effort to maintain.

We don't undo any existing svn symlinks if they exist since repo does
not care about them, and their existence doesn't hurt anything.

Bug: https://crbug.com/gerrit/15460
Change-Id: Ie8496b275bcc589771aa9f4ee874ed2ee6d5241d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327517
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-06 08:28:37 +00:00
c72bd8486a project: clean up now unused code
Now that we symlink worktree .git/ paths to .repo/projects/, we never
set share_refs=True anywhere, which means all of this logic is dead
code.  Throw it all away.  Do it as a separate commit to make the
parent commit easier to review.

Bug: https://crbug.com/gerrit/15273
Change-Id: If496d39029d3d3bd523ba24c603ce47a63ad9b51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/326817
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jack Neus <jackneus@google.com>
2022-01-06 04:08:05 +00:00
d53cb9549a sync: dropped "NOTICE: --use-superproject is in beta ..." message.
Tested the code with the following commands.

$ ./run_tests -v

Bug: [google internal] b/209511230
Change-Id: Ia3c6de47709f5276e324a5bb608383aba3b2c562
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327197
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-12-29 19:07:08 +00:00
cf0ba48649 sync: With --mirror option, don't display no-use-superproject... message.
+ Display 'Defaulting to no-use-superproject because there is no working tree.'
  message if --use-superproject option is used and we are not using
  superproject because manifest is either a mirror or is an archive.

Tested the code with the following commands.

$ ./run_tests -v

Tested the sync code by using repo_dev alias and pointing to this CL.

$ repo init -u https://android.googlesource.com/mirror/manifest --mirror

$ repo_dev sync
Receiving objects: 100% (3/3), done.eiving objects:  33% (1/3)

$ repo_dev sync --use-superproject
Defaulting to no-use-superproject because there is no working tree.
Fetching:  0% (0/2158) warming up

Bug: https://crbug.com/gerrit/15368
Change-Id: I16b87ee9623315dbc3100b612b1decdaab7ac1dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/325797
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-12-07 16:46:41 +00:00
2a089cfee4 project: migrate worktree .git/ dirs to symlinks
Historically we created a .git/ subdir in each source checkout and
symlinked individual files to the .repo/projects/ paths.  This layer
of indirection isn't actually needed: the .repo/projects/ paths are
guaranteed to only ever have a 1-to-1 mapping with the actual git
checkout.  So we don't need to worry about having files in .git/ be
isolated.

To that end, change how we manage the actual project checkouts from
a dir full of symlinks (and a few files) to a symlink to the internal
.repo/projects/ dir.  This makes the code simpler & faster.

The directory structure we have today is:
.repo/
  project-objects/chromiumos/third_party/kernel.git/
    <paths omitted as not relevant to this change>
  projects/src/third_party/kernel/
    v3.8.git/
      config
      description   -> …/project-objects/…/config
      FETCH_HEAD
      HEAD
      hooks/        -> …/project-objects/…/hooks/
      info/         -> …/project-objects/…/info/
      logs/
      objects/      -> …/project-objects/…/objects/
      packed-refs
      refs/
      rr-cache/     -> …/project-objects/…/rr-cache/
src/third_party/kernel/
  v3.8/
    .git/
      config        -> …/projects/…/v3.8.git/config
      description   -> …/project-objects/…/v3.8.git/description
      HEAD
      hooks/        -> …/project-objects/…/v3.8.git/hooks/
      index
      info/         -> …/project-objects/…/v3.8.git/info/
      logs/         -> …/projects/…/v3.8.git/logs/
      objects/      -> …/project-objects/…/v3.8.git/objects/
      packed-refs   -> …/projects/…/v3.8.git/packed-refs
      refs/         -> …/projects/…/v3.8.git/refs/
      rr-cache/     -> …/project-objects/…/v3.8.git/rr-cache/

The directory structure we have after this commit:
.repo/
  <nothing changes>
src/third_party/kernel/
  v3.8/
    .git            -> …/projects/…/v3.8.git

Bug: https://crbug.com/gerrit/15273
Change-Id: I9dd8def23fbfb2f4cb209a93f8b1b2b24002a444
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323695
Reviewed-by: Mike Nichols <mikenichols@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-12-01 15:27:16 +00:00
4a478edb44 init, sync: fixed flake8 warnings.
Tested:
+ run_tests
+ flake8 subcmds/init.py
+ flake8 subcmds/sync.py

Change-Id: Ie337481d8a210bfc49b0745f75c05a308a0e74d3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/324155
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-18 16:22:40 +00:00
6bd89aa657 superproject: Inherit --no-use-superproject with --mirror option.
init.py
+ Similar to opt.archive, gave an error if --mirror option is
  used with --use-superproject.

sync.py
+ Defaulted to --no-use-superproject if manifest is a mirror or
  archive (similar to error at line# 1067).

Tested:
+ run_tests
+ flake8 (will fix known errors in another CL).

$ repo_dev init -u sso://googleplex-android.git.corp.google.com/platform/manifest --use-superproject --mirror
Usage: repo init [options] [manifest url]

main.py: error: --mirror and --use-superproject cannot be used together.

+ repo init and repo sync with --mirror and without --mirror
  options.
  $ repo_dev init -u https://android.googlesource.com/platform/manifest
  $ repo_dev sync
    ...superproject.git: Initial setup for superproject completed.

+ With --mirror option, verfied there are no exceptions in git_superproject.py

Bug: [google internal] b/206537893
Change-Id: I059f20e76f0ab36f0587f29779bb53ede4663bd4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323955
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-18 01:27:41 +00:00
9c1fc5bc5d sync: Handle tag ref in "upstream" field
repo sync only handles a git tag properly when it is in the "revision"
field. However, "revision locked manifests" (`repo manifest
--revision-as-HEAD`) specifies the tag in the "upstream" field. The
issue is that this tag is not fetched. Only the commit that the tag
points to is fetched. This cases issues as
self._CheckForImmutableRevision() runs and comes to the conclusion that
the tag was changed while in fact, it was just not fetched. This causes
a full sync.

File docs/manifest-format.md, section Element-project:
> Attribute upstream: Name of the Git ref in which a sha1 can be found.
Used when syncing a revision locked manifest in -c mode to avoid having
to sync the entire ref space. Project elements not setting their own
upstream will inherit this value.

Change-Id: I0507d3a5f30aee8920a9f820bafedb48dd5db554
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323620
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Robin Schneider <ypid@riseup.net>
2021-11-16 20:43:57 +00:00
333c0a499b project: init hooks in objdir only
objdir is the .repo/project-objects/ dir based on the remote path.
gitdir is the .repo/projects/ dir based on the local source checkout
path.  When we setup the gitdir, we symlink "hooks" to the one in the
objdir.  But when we go to initialize the hooks, we do it via gitdir.
There is a 1-to-many mapping from project-objects to projects, so
initializing via gitdir can be repetitive.  Collapse the hook init
logic to the objdir init path.

Change-Id: I828fca60ce6e125d6706c709cdb2797faa40aa50
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323815
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 19:50:18 +00:00
fdeb20f43f sync: link the internal-fs-layout doc into checkouts
This should make it easy to discover for people poking around .repo/.

Change-Id: Ie5051551f25127c0592df5e36efba7bb2263e5d4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323701
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 01:39:53 +00:00
bf40957b38 git-review: add config file
This is used by the `git review` tool that some people use.

Change-Id: I8dac4e1dad155109a05181deaec61e1a74857b1f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323698
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 01:39:36 +00:00
f9e81c922d SUBMITTING_PATCHES: link to commit message style docs
Change-Id: I2090ebc43fc1c816b941a53dd89dbedf7bc61289
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323696
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jack Neus <jackneus@google.com>
2021-11-15 01:39:16 +00:00
e6601067ed man: refresh pages
Change-Id: I3f2c3ad77c16a76276bba2954887ab9e7605661c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323516
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-12 17:30:45 +00:00
3001d6a426 help: fix grammar in help text
Bug: https://crbug.com/gerrit/14838
Change-Id: Ic5000921ba9a1baa086153630ebbb429e3d17642
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323515
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-12 17:30:32 +00:00
00c5ea3787 Fix typo for ValueError
which will cause error log like below:
NameError: name 'ValueErrorl' is not defined

Change-Id: I388886b7cf6d700e224c3847b7ba4ba4fe9c041d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323015
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: 彭杨益 <pyy101727@gmail.com>
2021-11-07 02:32:08 +00:00
0531a623e1 sync: make --prune the default
If a remote deletes a ref, and it points to an object that doesn't
exist locally, we can get into a bad state, and the only way for the
user to recover is to run `repo sync --prune` (and to know that is
the option they need).  The error message is not helpful:

fatal: bad object refs/remotes/cros/firmware-zork-13421.B-master
error: https://chromium.googlesource.com/chromiumos/platform/ec did not send all necessary objects

This situation can also come up when the remote renames refs in a
UNIX FS incompatible way.  For example, replacing refs/heads/foo
with refs/heads/foo/bar.

Also add a --no-prune option for users to disable the behavior.

Bug: https://issuetracker.google.com/203366450
Change-Id: Icf45d838a10938feb091d29800f7e49240830ec3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322956
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-05 20:13:30 +00:00
2273f46cb3 sync: fix --tags option
This has been broken since it was added where --tags was actually
the same as --no-tags.  Oddly, it was copied from init where the
logic is correct.

Bug: https://crbug.com/gerrit/12401
Change-Id: I15b89da1a655176a11bebc22573b25c728055328
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322955
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-05 20:13:02 +00:00
7b9b251a5e project: fix format string in error message
BUG=None

Change-Id: I0b195fd919c6db8cb3547e8d6f4c733f2bd4a535
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322735
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
2021-11-05 16:59:14 +00:00
6251729cb4 superproject: added 'implies -c' in the help of --use-superproject option.
sync.py: deleted unused import errno.

Tested:
$ ./run_tests
$ flake8 repo subcmds/sync.py

Bug: https://crbug.com/gerrit/15208
Change-Id: I2bb3098f5602ded3861e000100766041ad93b53d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322555
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-01 22:15:53 +00:00
11b30b91df Support more url schemes for getting standalone manifest
urllib.requests.urlopen also supports file, so call it unless the
scheme is 'gs'.  This adds http, https, and ftp support.

Change-Id: I3f215c3ebd8e6dee29ba14c7e79ed99d37287109
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322095
Reviewed-by: Michael Kelly <mkelly@arista.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Matt Story <mstory@arista.com>
2021-10-27 13:20:35 +00:00
198838599c fetch: Fix stderr handling for gsutil
Previously gsutil stderr was getting piped into stdout, which
yields bad results if there are non-fatal warnings in stderr.

Additionally, we should fail outright if gsutil fails (by adding
`check = True`) rather than fail later on when we try to sync to
a manifest that is in fact just a stderr dump.

BUG=none
TEST=manual runs with bad gs urls

Change-Id: Id71791d0c3f180bd0601ef2c783a8e8e4afa8f59
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321935
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-26 22:18:28 +00:00
282d0cae89 ssh: handle FileNotFoundError errors
If ssh isn't installed, it throws a distinct error we have to catch.

Bug: https://crbug.com/gerrit/15196
Change-Id: I0660e842c304ce7575f5cb100894d05fd65f9454
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322055
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-10-26 16:18:45 +00:00
03ff276cd7 sync: properly handle standalone manifests for sync command
sync should not attempt to sync the manifest project if it was
created from a standalone manifest. The current work around is to
run sync with --nmu.

BUG=none
TEST=manual runs

Change-Id: I2e121af0badf9642143e77c7af89d1c2d993b0f3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321195
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-15 17:20:00 +00:00
4ee4a45d03 subcmds/sync: Use pack-refs instead of gc for redundant gitdirs.
Previously `git gc` was being run on every gitdir even when they shared
the same objects. Instead only call it once and use pack-refs for the
gitdirs that were not gc'ed.

Bug: https://crbug.com/gerrit/15113
Test: repo sync -j # and check that git pack-refs is called
Change-Id: Icff37ab3ec78cfb44391d8cc7f2d875991532320
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320275
Tested-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-14 12:27:12 +00:00
0f6f16ed17 repo: more arg checking for --standalone-manifest re-inits
`repo init` doesn't do anything on re-init when the checkout has
been initialized using --standalone manifest. Rather than let the
tool run through its existing flows (which happen to noop), check
the args and explicitly quit if a bare `repo init` is run on a
standalone checkout.

BUG=none
TEST=manual tests

Change-Id: Ie4346ef6df1282ec3e3f8045a08138c93653fece
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320735
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-11 18:58:11 +00:00
76491590b8 repo: fix bug with --standalone-manifest
We were accidentally always setting manifest.standlone in config,
which was messing up behavior for standard use cases.

BUG=gerrit:15160
TEST=manual runs

Change-Id: Ic80f084ae97de5721aced3bb52d3ea9115f8d833
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320715
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-11 18:57:57 +00:00
6a74c91f50 sign-launcher: make the help text more automatic
Rather than display "3.0" all the time and confuse people, extract
the version from the launcher we're signing and display that.

Also reformat the text to follow our current practice: upload the
versioned launcher by itself first, and then later copy that over
the default.

And while we're here, add tips for rollbacks.

Change-Id: I1654425c88e5c67d78879f2f33ad685c59be14dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319637
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-10-06 17:02:56 +00:00
669efd0fd7 subcmds/sync: Disable autoDetach for git gc.
gc.autoDetach is enabled by default which makes 'git gc --auto' return
immediately and run in background. This can lead to a pile up of
operations all using large amounts of memory at the same time. To avoid
this set gc.autoDetach to false so that the garbage collect task waits
for instances to finish before spawning more.

Bug: https://crbug.com/gerrit/15113
Test: repo sync -j # and check the number of 'git gc' processes
Change-Id: Ic0815156ba3db03972968f33f6f9f51e4928f23b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319835
Tested-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-05 14:12:01 +00:00
a0f6006ae7 git_config: Fixed test.gitconfig getting updated when running tests.
Moved test_GetSyncAnalysisStateData to GitConfigReadWriteTests class.

Deleted [repo "syncstate*..] data from tests/fixtures/test.gitconfig.

Tested:
./run_tests
...
tests/test_git_config.py::GitConfigReadWriteTests::test_GetSyncAnalysisStateData PASSED [ 84%]
...

Bug: https://crbug.com/gerrit/15103
Change-Id: I8cb89ce10b025994a045106c9c66dd243ae8ba50
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319557
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-09-30 21:45:09 +00:00
2ddbf8a8bf Merge "Merge history of v2.14.5." into main 2021-09-30 21:37:09 +00:00
445723fd37 Merge history of v2.14.5.
The v2.14.[345] releases were cut on a branch based on v2.14.2.  We
had some regression fixes we wanted in v2.14, but too many risky
changes landed in main since to cut another v2.14.x directly, and
we didn't want to destabilize even more by pushing a v2.15 right
away.  So we branched to keep things healthy.

But people with old checkouts trying to upgrade from those versions
run into an old repo bug where it only selfupdates with fast-forwards,
and repo can't fast-forward from those divergent histories.  So let's
do a merge commit to stitch the history back together.

There's no actual changes in here.

Change-Id: I05a96048e3846321e57c5f5224fb8dcf3c191d35
2021-09-30 21:36:56 +00:00
436bde5137 Merge history of v1.13.11.
For older versions of repo, this would make it easier for it to perform
a self update by making it a fast-forward from the following tags:

  v1.13.9.2, v1.13.9.3, v1.13.9.4, v1.13.10, v1.13.11

Change-Id: Ia75776312eaf802a150db8bd7c0a6dce57914580
2021-09-30 11:49:08 -07:00
4f88206178 trace2_event: Add remove_prefix to fix failing tests on Linux & macOS.
removeprefix is available i python 3.9. Mac and Linux are running in
a version below 3.9. Thus tests are failing with the following error:
  "AttributeError: 'str' object has no attribute 'removeprefix' "

Replaced the removeprefix with custom function which we will delete
once Linux and macOS versions are updated.

Tested:
$ ./run_tests

Bug: [google internal] b/201453085
Change-Id: I9b4d564ff1176e1b4471805ef05472c1914cd9f9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319375
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-09-29 21:24:59 +00:00
f88282ccc2 git_config: update error handling with no config file
Now that _do throws an exception when `git` fails, update the logic
that tries to read config files but the file doesn't exist.

Bug: b/192664812
Change-Id: I6417ecd70891b8f2d5f2bdb819f91df69ac4b70c
Test: `repo upload` no longer crashes when .repo/config doesn't exist
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319295
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-09-29 01:02:47 +00:00
8967a5aec6 launcher: bump version for new release
Change-Id: I9812185c9dfc11289547f5956c0cbe567d720f7f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319335
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-09-28 20:54:56 +00:00
2f3c3316e4 Update revisionId if required when using extend-project
When a hard revision ID is provided in a regular project tag then the
revisionId is updated as well if it is a commit hash.  The difference
is that if the revisionExpr is a commit, git-repo needs to update
refs/remotes/m/master with update-ref not symbolic-ref, as the latter
must refer to another ref, not to a specific commit.

Change-Id: I215a62dabb30225e480ad2c731416d775fc0c750
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310963
Tested-by: Michael Kelly <mkelly@arista.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-09-28 20:12:00 +00:00
37c21c268b Add 'dest-path' to extend-project to support changing path
This allows us to move the repository to a new location in the source
tree without having to remove-project + add a new project tag.

Change-Id: I4dba6151842e57f6f2b8fe60cda260ecea68b7b4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310962
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Michael Kelly <mkelly@arista.com>
2021-09-28 20:12:00 +00:00
b12c369e0b superproject: Only trigger enrollment logic when manifest have it.
The current code would check for enrollment status when the user did not
explicitly specify --[no-]use-superproject and do not have a remembered
value in their repo client. However, because superproject only makes
sense for manifests that have one specified, we should skip the
enrollment logic in that case.

Address this by checking manifest.superproject prior to proceeding. This
would avoid showing the greeting message of superproject enrollment
which can be confusing for developers.

Tested:
  For manifest without superproject:
   - repo sync --use-superproject will still show message for
     superproject;
   - repo sync will not show message regardless of enrollment state
  For manifest with superproject:
   - repo sync will show message and perform enrollment if not
     previously enrolled

Bug: https://crbug.com/gerrit/15039
Change-Id: Ic2be9f9d037f0e7cf3446da474a5a0d0e4bd88da
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319255
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-09-28 19:42:01 +00:00
bbe8836494 superproject: Log syncstate's parameter as data-json it it is an array.
All the values of syncstate are strings, check the first byte and last
byte to see if it is an array. For syncstate data, there were no false
positives.

Tested:
$ repo_dev sync

Verified event logged for argv is "data-json".

$./run_tests

Bug: [google internal] b/201102002
Change-Id: Id56adb532b80267f08d09147ac663cdd5987ce87
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319075
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-09-28 18:22:49 +00:00
9d96f58f5f make file removal a bit more robust
Some of the file removal calls are subject to race conditions (if
something else deletes the file), so extend our remove API to have
an option to ignore ENOENT errors.  Then update a bunch of random
call sites to use this new functionality.

Change-Id: I31a9090e135452033135337a202a4fc2dbf8b63c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319195
Reviewed-by: Sean McAllister <smcallis@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-09-28 16:06:50 +00:00
7a1e7e772f repo: add support for reading standalone manifests from disk
BUG=b:192664812
TEST=existing tests (no coverage), manual runs

Change-Id: Ic032417ecfca77d5e0de1b1ff62b30ce8205bfc5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318715
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-09-28 16:03:21 +00:00
c474c9cba1 repo: Add support for standalone manifests
Added --standalone_manifest to repo tool. If set, the
manifest is downloaded directly from the appropriate source
(currently, we only support GS) and used instead of creating
a manifest git checkout. The manifests.git repo is still created to
keep track of various config but is marked as being for a standalone
manifest so that the repo tool doesn't try to run networked git
commands in it.

BUG=b:192664812
TEST=existing tests (no coverage), manual runs

Change-Id: I84378cbc7f8e515eabeccdde9665efc8cd2a9d21
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312942
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-09-28 15:40:46 +00:00
956f7363d1 superproject: Log argv parameter of syncstate as 'data-json'.
Fixed: "we need to make a special case for logging the argv; it
should probably be a "data-json" event so that we log this directly as
an array rather than an encoded string.

Tested:
$ repo_dev sync

Verified event logged for argv is "data-json".

$./run_tests

Bug: [google internal] b/201102002
Change-Id: I18ccec79c73c8dc931cb8afc472b2361db8aea4c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319055
Reviewed-by: Josh Steadmon <steadmon@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-09-27 19:11:14 +00:00
6f8c1bf4ff Fix indent error which would have prevented choice expiration to work.
Change-Id: I077e05eea23ad58d1dde2c9fe5608660a56d03e5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318815
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Amith Dsouza <amithds@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
2021-09-27 18:59:21 +00:00
e0b16a22a0 superproject: support a new revision attribute.
Tested:
$ ./run_tests

Verified that a manifest that specified superproject revision would use
the specified revision, and superproject will use the default revision.

Note that this is a slight behavior change from earlier repo versions,
which would always use the branch name of the manifest itself. However,
the new behavior would be more consisitent with regular "project"
element and would allow superproject be used even if it is not enabled
for the particular manifest branch, so we have decided to make the
change as it would provide more flexibility and better matches what
other elements would do.

Bug: [google internal] b/187868160
Change-Id: I35255ee347aff6e65179f7879d52931f168b477e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317643
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-09-27 06:36:05 +00:00
d669d2dee5 release-process: update distro baseline & add OpenSSH
Stop tracking Ubuntu Trusty & Xenial and Debian Jessie & Stretch
as they only had Python 3.5 available which we've dropped.

Backfill OpenSSH versions since we've started testing for it.

Change-Id: I03183ed97f6e43dce8a00e36cce2956544a26afc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318835
Reviewed-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-09-24 16:42:56 +00:00
366824937c platform_utils: os.rename exception when src and des on different file system
Symptom: repo sync exception
Root Cause: os.rename only works when source and destination are on the same file system
Solution: using shutil.move

to save disk usage, I create links for projects and project-objects, link to folder on another disk
lrwxrwxrwx  1 owenwen owenwen   47 Jun  9 16:40 project-objects -> /disk3/AndroidLocalRepos/.repo/project-objects/
lrwxrwxrwx  1 owenwen owenwen   40 Jun  9 16:40 projects -> /disk3/AndroidLocalRepos/.repo/projects/

below are exception I met:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/disk2/Android11/.repo/repo/subcmds/sync.py", line 550, in _CheckoutOne
    project.Sync_LocalHalf(syncbuf, force_sync=force_sync)
  File "/disk2/Android11/.repo/repo/project.py", line 1251, in Sync_LocalHalf
    self._InitWorkTree(force_sync=force_sync, submodules=submodules)
  File "/disk2/Android11/.repo/repo/project.py", line 2801, in _InitWorkTree
    self._CheckDirReference(self.gitdir, dotgit, share_refs=True)
  File "/disk2/Android11/.repo/repo/project.py", line 2674, in _CheckDirReference
    platform_utils.rename(dst_path, src_path)
  File "/disk2/Android11/.repo/repo/platform_utils.py", line 127, in rename
    os.rename(src, dst)
OSError: [Errno 18] Invalid cross-device link: '/disk2/Android11/system/libhidl/.git/packed-refs' -> '/disk2/Android11/.repo/projects/system/libhidl.git/packed-refs'
"""

Change-Id: Ifda2f16530cc5a8f280169f482ee858f9e5241d3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/316002
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-09-24 08:20:06 +00:00
a84f43a006 manifest: make repo-hooks more robust wrt element ordering
Currently, repo will fail to sync to a manifest if the definition
of the repo-hooks project comes after the repo-hooks element.

BUG=none
TEST=new test, run_tests

Change-Id: I0bf85625173492af6c6404d4b67543e96e670562
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318520
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jack Neus <jackneus@google.com>
2021-09-23 21:17:38 +00:00
0468feac39 update-manpages: avoid regen just for datestamp update
To avoid noise due to the passage of time, don't regenerate man pages
if the only thing different is the datestamp in the header.

Change-Id: Ic8d7b08d12e59c66994c0cc2d4ec2d2ed3eb6e6d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318575
Reviewed-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-09-22 19:37:35 +00:00
0ec2029833 superproject: Move enrollment to opt-out when enabled globally
Our internal experiments was a success so far and we are enrolling 100%
users now.  Instead of asking every two weeks, simply consider a lack of
unexpired choice as accepting the system default.

With this change the user would still be able to override the system
default with --no-use-superproject, or to permanently set the choice in
user's profile with git config --global repo.superprojectchoice.

Bug: [google internal] b/190688390
Change-Id: Idc77a9cbf88a169d90304169e91f0d722dc4ac8b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317975
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
2021-09-20 07:21:22 +00:00
d8e8ae8990 superproject: Log branch and remote url with every log message.
Saved superproject's remote URL in _remote_url data and used it
in the _Fecth function.

Tested:
$ ./run_tests
$ flake8 git_superproject.py
$ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest
$ repo_dev sync

   Verified the all log messages have the following format.
   repo superproject branch: <branch> url: <url> warning: <message>

Bug: [google internal] b/200072098
Change-Id: Iac6af7c99225479fd50bc6909396b22e0ce5f76b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318177
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-09-16 15:46:19 +00:00
6448a4f2af sync: Log repo sync state events as 'data' events.
git_trace2_event_log.py:
+ Added LogDataConfigEvents method to log 'data' events.
  Sync's current_sync_state and previous_sync_state are logged
  as 'data' events in the current log.

  It logs are key/value in the |config| argument. Each key is
  prefixed with |prefix| argument.

  The following are sample events that are logged during repo sync.

   {"event":"data",
   "sid":"repo-20210914T181545Z-P000330c0/repo-20210914T181545Z-P000330c0",
   "thread":"MainThread",
   "time":"2021-09-14T18:16:19.935846Z",
   "key":"previous_sync_state/repo.syncstate.main.synctime",
   "value":"2021-09-14T17:27:11.573717Z"}

   {"event":"data",
   "sid":"repo-20210914T181545Z-P000330c0/repo-20210914T181545Z-P000330c0",
   "thread":"MainThread",
   "time":"2021-09-14T18:16:19.955546Z",
   "key":"current_sync_state/repo.syncstate.main.synctime",
   "value":"2021-09-14T18:16:19.935979Z"}

tests/test_git_trace2_event_log.py:
+ Added unit tests

sync.py:
+ Changed logging calls to LogDataConfigEvents.

Tested:
$ ./run_tests

Tested it by running the following command multiple times.
$ repo_dev sync -j 20
  repo sync has finished successfully

  Verified config data is looged in trace2 event logs.

Bug: [google internal] b/199758376
Change-Id: I75fd830e90c1811ec28510538c99a2632b104e85
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317823
Reviewed-by: Josh Steadmon <steadmon@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-09-14 21:36:12 +00:00
1328c35a4d superproject: Provide accurate feedback for user choice
Currently the code would give a message that would appear like the user
have enrolled the experiment regardless of the actual choice. For users
who choose to not enroll in the experiment, we should give them
instructions to override (enable) superproject once instead of how to
disable it, which is what the code already behave.

Bug: [google internal] b/199167992
Change-Id: Iba3314cb510aedf024375a26baa8bc1d5e2846cf
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317382
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
2021-09-08 20:35:42 +00:00
148e1ce81a sync: fix recursive fetching
Commit b2fa30a2b8 ("sync: switch network
fetch to multiprocessing") accidentally changed the variable passed to
the 2nd fetch call from |missing| to |to_fetch| due to a copy & paste
of the earlier changed logic.  Undo that to fix git submodule fetching.

Bug: https://crbug.com/gerrit/14489
Change-Id: I627954f80fd2e80d9d5809b530aa6b0ef9260abb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305262
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-05-04 22:43:09 -04:00
32ca6687ae git_config: hoist Windows ssh check earlier
The ssh master logic has never worked under Windows which is why this
code always returned False when running there (including cygwin).  But
the OS check was still done while holding the threading lock.  While
it might be a little slower than necessary, it still worked.

The switch from the threading module to the multiprocessing module
changed global behavior subtly under Windows and broke things: the
globals previously would stay valid, but now they get cleared.  So
the lock is reset to None in children workers.

We could tweak the logic to pass the lock through, but there isn't
much point when the rest of the code is still disabled in Windows.
So perform the platform check before we grab the lock.  This fixes
the crash, and probably speeds things up a few nanoseconds.

This shouldn't be a problem on Linux systems as the platform fork
will duplicate the existing process memory (including globals).

Bug: https://crbug.com/gerrit/14480
Change-Id: I1d1da82c6d7bd6b8cdc1f03f640a520ecd047063
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305149
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-05-04 19:49:58 -04:00
0ae9503a86 sync: fix print error when handling server error
When converting this logic from print() to the output buffer, this
error codepath should have dropped the use of the file= redirect.

Bug: https://crbug.com/gerrit/14482
Change-Id: Ib484924a2031ba3295c1c1a5b9a2d816b9912279
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/305142
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-05-04 12:48:42 -04:00
d92076d930 Revert "Save cookies back to jar when fetching clone.bundle"
This reverts commit 4abf8e6ef8.

The curl process for updating the cookie file is not atomic.  When
fetching many bundles in parallel, we can sometimes corrupt the file
causing it to be cleared.  Since users should manage gitcookies on
their own, leave it read-only.

Bug: https://crbug.com/gerrit/12300
Change-Id: Id472c99b197bc4cf8533c649f8881509f38643c1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254092
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
(cherry picked from commit dc1d0e0c7f)
2020-02-11 21:03:35 -05:00
aeb2eee9d3 repo: bump launcher version
This way we can push out the updated stable branch change.

Change-Id: I72d5dab4523a10dfeb6529796892096aa80eba3c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254492
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2020-02-12 00:12:14 +00:00
45d1c372a7 project: fix bytes/str encoding when updating git submodules
Since tempfile.mkstemp() returns a file handle in binary mode,
make sure we turn our strings into bytes before writing.

Bug: https://crbug.com/gerrit/12043
Change-Id: I3e84d595e84b8bc12a1fbc7fd0bb3ea0ba2832b0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254393
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
(cherry picked from commit 163d42eb43)
2020-02-11 14:46:13 -05:00
19607b2817 repo: allow REPO_REV to be an env var
We do this for REPO_URL already.

Bug: https://crbug.com/gerrit/10233
Change-Id: I53410645474b00d900467c96fa5d8446f3a607d3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253552
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
(cherry picked from commit 563f1a6512)
2020-02-11 14:44:29 -05:00
68744dbc01 Fixing forall subcommand for Py3
Execution of 'repo forall -p -c' doesn't work with Py3 and ends up
with an error:

Got an error, terminating the pool: TypeError: can only concatenate
str (not "bytes") to str

That's fixed by using the decode() method.

Change-Id: Ice01aaa1822dde8d957b5bf096021dd5a2b7dd51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253659
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Jiri Tyr <jiri.tyr@gmail.com>
(cherry picked from commit 83a3227b62)
2020-02-10 23:31:45 -05:00
ef412624e9 remove spurious +x bits
These files are not directly executable, so drop the +x bits.

Change-Id: Iaf19a03a497686cc21103e7ddf08073173440dd1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254076
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
(cherry picked from commit e7c91889a6)
2020-02-10 23:31:03 -05:00
a06ab7d28b find python via env
This allows these scripts to run through the active version of the
virtualenv python when invoked via tox.

Change-Id: Ib52f475b7b20c34d62cfd179a1341da1a08a8b5c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253974
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
(cherry picked from commit 1b117db767)
2020-02-10 23:30:58 -05:00
471a7ed5f7 git_config: fix encoding handling in GetUrlCookieFile
Make sure we decode the bytes coming from the subprocess.Popen as
we're treating them as strings.

Change-Id: I44100ca5cd94f68a35d489936292eb641006edbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253973
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
(cherry picked from commit ded477dbb9)
2020-02-10 23:28:40 -05:00
619a2b5887 Fix inverted logic around [gitc-]init and -c
Instead of not using '-c' for '--current-branch' when using gitc, we
were only using '-c' when using gitc, so we still had the conflict with
the gitc option, and other users still couldn't use '-c'.

Test: repo init -u https://android.googlesource.com/platform/manifest; repo init -c
Test: repo gitc-init -u ... -b ... -c testing
Change-Id: I71e4950a49c281418249f0783c6a2ea34f0d3e2b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253795
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Dan Willemsen <dwillemsen@google.com>
(cherry picked from commit 93293ca47f)
2020-02-07 15:54:52 -05:00
ab15e42fa4 Do not try to fetch default revision for mirrors always
* Mirrors may contain multiple projects, some of which may not
  always contain the default revision.
* Only fetch the default revision explicitly if
  '--current-branch' is set.
* Fixes breakage casued by
  commit 6856f98467
  "Fix repo mirror with --current-branch"

Bug: https://crbug.com/gerrit/12274
Change-Id: Iaafabe2992f76f3644b841f24245d3e19c9515a9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253093
Reviewed-by: Kuang-che Wu <kcwu@chromium.org>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Chirayu Desai <chirayudesai1@gmail.com>
(cherry picked from commit f7b64e3350)
2020-02-06 09:19:35 -05:00
75c02fe4cb init: handle -c conflicts with gitc-init
We keep getting requests for init to support -c.  This conflicts with
gitc-init which allocates -c for its own use.  Lets make this dynamic
so we keep it with "init" but omit it for "gitc-init".

Bug: https://crbug.com/gerrit/10200
Change-Id: Ibf69c2bbeff638e28e63cb08926fea0c622258db
(cherry picked from commit 66098f707a)
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253392
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-02-05 18:04:11 +00:00
afd1b4023f repo: point default branch to repo-1
Since this will be feature-frozen for Python 2 users, lets point the
default update branch to "repo-1" rather than "stable" as the latter
will follow the master development (and Python 3-only).

Bug: https://crbug.com/gerrit/10418
Change-Id: Iceff0983684a580dc5c9ec1c60acfb5eda5ce2c4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253172
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2020-02-05 02:56:08 +00:00
32 changed files with 1045 additions and 437 deletions

5
.gitreview Normal file
View File

@ -0,0 +1,5 @@
[gerrit]
host=gerrit-review.googlesource.com
scheme=https
project=git-repo.git
defaultbranch=main

View File

@ -3,7 +3,7 @@
# Short Version
- Make small logical changes.
- Provide a meaningful commit message.
- [Provide a meaningful commit message][commit-message-style].
- Check for coding errors and style nits with flake8.
- Make sure all code is under the Apache License, 2.0.
- Publish your changes for review.
@ -26,10 +26,11 @@ yourself with the following relevant bits.
## Make separate commits for logically separate changes.
Unless your patch is really trivial, you should not be sending
out a patch that was generated between your working tree and your
commit head. Instead, always make a commit with complete commit
message and generate a series of patches from your repository.
Unless your patch is really trivial, you should not be sending out a patch that
was generated between your working tree and your commit head.
Instead, always make a commit with a complete
[commit message][commit-message-style] and generate a series of patches from
your repository.
It is a good discipline.
Describe the technical detail of the change(s).
@ -171,3 +172,6 @@ After you receive a Code-Review+2 from the maintainer, select the Verified
button on the gerrit page for the change. This verifies that you have tested
your changes and notifies the maintainer that they are ready to be submitted.
The maintainer will then submit your changes to the repository.
[commit-message-style]: https://chris.beams.io/posts/git-commit/

View File

@ -157,6 +157,7 @@ User controlled settings are initialized when running `repo init`.
| Setting | `repo init` Option | Use/Meaning |
|------------------- |---------------------------|-------------|
| manifest.groups | `--groups` & `--platform` | The manifest groups to sync |
| manifest.standalone | `--standalone-manifest` | Download manifest as static file instead of creating checkout |
| repo.archive | `--archive` | Use `git archive` for checkouts |
| repo.clonebundle | `--clone-bundle` | Whether the initial sync used clone.bundle explicitly |
| repo.clonefilter | `--clone-filter` | Filter setting when using [partial git clones] |

View File

@ -90,6 +90,7 @@ following DTD:
<!ELEMENT extend-project EMPTY>
<!ATTLIST extend-project name CDATA #REQUIRED>
<!ATTLIST extend-project path CDATA #IMPLIED>
<!ATTLIST extend-project dest-path CDATA #IMPLIED>
<!ATTLIST extend-project groups CDATA #IMPLIED>
<!ATTLIST extend-project revision CDATA #IMPLIED>
<!ATTLIST extend-project remote CDATA #IMPLIED>
@ -103,8 +104,9 @@ following DTD:
<!ATTLIST repo-hooks enabled-list CDATA #REQUIRED>
<!ELEMENT superproject EMPTY>
<!ATTLIST superproject name CDATA #REQUIRED>
<!ATTLIST superproject remote IDREF #IMPLIED>
<!ATTLIST superproject name CDATA #REQUIRED>
<!ATTLIST superproject remote IDREF #IMPLIED>
<!ATTLIST superproject revision CDATA #IMPLIED>
<!ELEMENT contactinfo EMPTY>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
@ -336,6 +338,11 @@ against changes to the original manifest.
Attribute `path`: If specified, limit the change to projects checked out
at the specified path, rather than all projects with the given name.
Attribute `dest-path`: If specified, a path relative to the top directory
of the repo client where the Git working directory for this project
should be placed. This is used to move a project in the checkout by
overriding the existing `path` setting.
Attribute `groups`: List of additional groups to which this project
belongs. Same syntax as the corresponding element of `project`.
@ -432,6 +439,11 @@ same meaning as project's name attribute. See the
Attribute `remote`: Name of a previously defined remote element.
If not supplied the remote given by the default element is used.
Attribute `revision`: Name of the Git branch the manifest wants
to track for this superproject. If not supplied the revision given
by the remote element is used if applicable, else the default
element is used.
### Element contactinfo
***

View File

@ -208,85 +208,132 @@ Things in bold indicate stuff to take note of, but does not guarantee that we
still support them.
Things in italics are things we used to care about but probably don't anymore.
| Date | EOL | [Git][rel-g] | [Python][rel-p] | [Ubuntu][rel-u] / [Debian][rel-d] | Git | Python |
|:--------:|:------------:|--------------|-----------------|-----------------------------------|-----|--------|
| Oct 2008 | *Oct 2013* | | 2.6.0 | *10.04 Lucid* - 10.10 Maverick / *Squeeze* |
| Date | EOL | [Git][rel-g] | [Python][rel-p] | [SSH][rel-o] | [Ubuntu][rel-u] / [Debian][rel-d] | Git | Python | SSH |
|:--------:|:------------:|:------------:|:---------------:|:------------:|-----------------------------------|-----|--------|-----|
| Apr 2008 | | | | 5.0 |
| Jun 2008 | | | | 5.1 |
| Oct 2008 | *Oct 2013* | | 2.6.0 | | *10.04 Lucid* - 10.10 Maverick / *Squeeze* |
| Dec 2008 | *Feb 2009* | | 3.0.0 |
| Feb 2009 | *Mar 2012* | | | Debian 5 Lenny | 1.5.6.5 | 2.5.2 |
| Jun 2009 | *Jun 2016* | | 3.1.0 | *10.04 Lucid* - 10.10 Maverick / *Squeeze* |
| Feb 2010 | *Oct 2012* | 1.7.0 | | *10.04 Lucid* - *12.04 Precise* - 12.10 Quantal |
| Apr 2010 | *Apr 2015* | | | *10.04 Lucid* | 1.7.0.4 | 2.6.5 3.1.2 |
| Jul 2010 | *Dec 2019* | | **2.7.0** | 11.04 Natty - **<current>** |
| Oct 2010 | | | | 10.10 Maverick | 1.7.1 | 2.6.6 3.1.3 |
| Feb 2011 | *Feb 2016* | | | Debian 6 Squeeze | 1.7.2.5 | 2.6.6 3.1.3 |
| Apr 2011 | | | | 11.04 Natty | 1.7.4 | 2.7.1 3.2.0 |
| Oct 2011 | *Feb 2016* | | 3.2.0 | 11.04 Natty - 12.10 Quantal |
| Oct 2011 | | | | 11.10 Ocelot | 1.7.5.4 | 2.7.2 3.2.2 |
| Apr 2012 | *Apr 2019* | | | *12.04 Precise* | 1.7.9.5 | 2.7.3 3.2.3 |
| Sep 2012 | *Sep 2017* | | 3.3.0 | 13.04 Raring - 13.10 Saucy |
| Oct 2012 | *Dec 2014* | 1.8.0 | | 13.04 Raring - 13.10 Saucy |
| Oct 2012 | | | | 12.10 Quantal | 1.7.10.4 | 2.7.3 3.2.3 |
| Apr 2013 | | | | 13.04 Raring | 1.8.1.2 | 2.7.4 3.3.1 |
| May 2013 | *May 2018* | | | Debian 7 Wheezy | 1.7.10.4 | 2.7.3 3.2.3 |
| Oct 2013 | | | | 13.10 Saucy | 1.8.3.2 | 2.7.5 3.3.2 |
| Feb 2014 | *Dec 2014* | **1.9.0** | | **14.04 Trusty** |
| Mar 2014 | *Mar 2019* | | **3.4.0** | **14.04 Trusty** - 15.10 Wily / **Jessie** |
| Apr 2014 | **Apr 2022** | | | **14.04 Trusty** | 1.9.1 | 2.7.5 3.4.0 |
| Feb 2009 | | | | 5.2 |
| Feb 2009 | *Mar 2012* | | | | Debian 5 Lenny | 1.5.6.5 | 2.5.2 |
| Jun 2009 | *Jun 2016* | | 3.1.0 | | *10.04 Lucid* - 10.10 Maverick / *Squeeze* |
| Sep 2009 | | | | 5.3 | *10.04 Lucid* |
| Feb 2010 | *Oct 2012* | 1.7.0 | | | *10.04 Lucid* - *12.04 Precise* - 12.10 Quantal |
| Mar 2010 | | | | 5.4 |
| Apr 2010 | | | | 5.5 | 10.10 Maverick |
| Apr 2010 | *Apr 2015* | | | | *10.04 Lucid* | 1.7.0.4 | 2.6.5 3.1.2 | 5.3 |
| Jul 2010 | *Dec 2019* | | *2.7.0* | | 11.04 Natty - *<current>* |
| Aug 2010 | | | | 5.6 |
| Oct 2010 | | | | | 10.10 Maverick | 1.7.1 | 2.6.6 3.1.3 | 5.5 |
| Jan 2011 | | | | 5.7 |
| Feb 2011 | | | | 5.8 | 11.04 Natty |
| Feb 2011 | *Feb 2016* | | | | Debian 6 Squeeze | 1.7.2.5 | 2.6.6 3.1.3 |
| Apr 2011 | | | | | 11.04 Natty | 1.7.4 | 2.7.1 3.2.0 | 5.8 |
| Sep 2011 | | | | 5.9 | *12.04 Precise* |
| Oct 2011 | *Feb 2016* | | 3.2.0 | | 11.04 Natty - 12.10 Quantal |
| Oct 2011 | | | | | 11.10 Ocelot | 1.7.5.4 | 2.7.2 3.2.2 | 5.8 |
| Apr 2012 | | | | 6.0 | 12.10 Quantal |
| Apr 2012 | *Apr 2019* | | | | *12.04 Precise* | 1.7.9.5 | 2.7.3 3.2.3 | 5.9 |
| Aug 2012 | | | | 6.1 | 13.04 Raring |
| Sep 2012 | *Sep 2017* | | 3.3.0 | | 13.04 Raring - 13.10 Saucy |
| Oct 2012 | *Dec 2014* | 1.8.0 | | | 13.04 Raring - 13.10 Saucy |
| Oct 2012 | | | | | 12.10 Quantal | 1.7.10.4 | 2.7.3 3.2.3 | 6.0 |
| Mar 2013 | | | | 6.2 | 13.10 Saucy |
| Apr 2013 | | | | | 13.04 Raring | 1.8.1.2 | 2.7.4 3.3.1 | 6.1 |
| May 2013 | *May 2018* | | | | Debian 7 Wheezy | 1.7.10.4 | 2.7.3 3.2.3 |
| Sep 2013 | | | | 6.3 |
| Oct 2013 | | | | | 13.10 Saucy | 1.8.3.2 | 2.7.5 3.3.2 | 6.2 |
| Nov 2013 | | | | 6.4 |
| Jan 2014 | | | | 6.5 |
| Feb 2014 | *Dec 2014* | **1.9.0** | | | *14.04 Trusty* |
| Mar 2014 | *Mar 2019* | | *3.4.0* | | *14.04 Trusty* - 15.10 Wily / *Jessie* |
| Mar 2014 | | | | 6.6 | *14.04 Trusty* - 14.10 Utopic |
| Apr 2014 | *Apr 2022* | | | | *14.04 Trusty* | 1.9.1 | 2.7.5 3.4.0 | 6.6 |
| May 2014 | *Dec 2014* | 2.0.0 |
| Aug 2014 | *Dec 2014* | **2.1.0** | | 14.10 Utopic - 15.04 Vivid / **Jessie** |
| Oct 2014 | | | | 14.10 Utopic | 2.1.0 | 2.7.8 3.4.2 |
| Aug 2014 | *Dec 2014* | *2.1.0* | | | 14.10 Utopic - 15.04 Vivid / *Jessie* |
| Oct 2014 | | | | 6.7 | 15.04 Vivid |
| Oct 2014 | | | | | 14.10 Utopic | 2.1.0 | 2.7.8 3.4.2 | 6.6 |
| Nov 2014 | *Sep 2015* | 2.2.0 |
| Feb 2015 | *Sep 2015* | 2.3.0 |
| Mar 2015 | | | | 6.8 |
| Apr 2015 | *May 2017* | 2.4.0 |
| Apr 2015 | **Jun 2020** | | | **Debian 8 Jessie** | 2.1.4 | 2.7.9 3.4.2 |
| Apr 2015 | | | | 15.04 Vivid | 2.1.4 | 2.7.9 3.4.3 |
| Jul 2015 | *May 2017* | 2.5.0 | | 15.10 Wily |
| Apr 2015 | *Jun 2020* | | | | *Debian 8 Jessie* | 2.1.4 | 2.7.9 3.4.2 |
| Apr 2015 | | | | | 15.04 Vivid | 2.1.4 | 2.7.9 3.4.3 | 6.7 |
| Jul 2015 | *May 2017* | 2.5.0 | | | 15.10 Wily |
| Jul 2015 | | | | 6.9 | 15.10 Wily |
| Aug 2015 | | | | 7.0 |
| Aug 2015 | | | | 7.1 |
| Sep 2015 | *May 2017* | 2.6.0 |
| Sep 2015 | **Sep 2020** | | **3.5.0** | **16.04 Xenial** - 17.04 Zesty / **Stretch** |
| Oct 2015 | | | | 15.10 Wily | 2.5.0 | 2.7.9 3.4.3 |
| Jan 2016 | *Jul 2017* | **2.7.0** | | **16.04 Xenial** |
| Sep 2015 | *Sep 2020* | | *3.5.0* | | *16.04 Xenial* - 17.04 Zesty / *Stretch* |
| Oct 2015 | | | | | 15.10 Wily | 2.5.0 | 2.7.9 3.4.3 | 6.9 |
| Jan 2016 | *Jul 2017* | *2.7.0* | | | *16.04 Xenial* |
| Feb 2016 | | | | 7.2 | *16.04 Xenial* |
| Mar 2016 | *Jul 2017* | 2.8.0 |
| Apr 2016 | **Apr 2024** | | | **16.04 Xenial** | 2.7.4 | 2.7.11 3.5.1 |
| Jun 2016 | *Jul 2017* | 2.9.0 | | 16.10 Yakkety |
| Apr 2016 | *Apr 2024* | | | | *16.04 Xenial* | 2.7.4 | 2.7.11 3.5.1 | 7.2 |
| Jun 2016 | *Jul 2017* | 2.9.0 | | | 16.10 Yakkety |
| Jul 2016 | | | | 7.3 | 16.10 Yakkety |
| Sep 2016 | *Sep 2017* | 2.10.0 |
| Oct 2016 | | | | 16.10 Yakkety | 2.9.3 | 2.7.11 3.5.1 |
| Nov 2016 | *Sep 2017* | **2.11.0** | | 17.04 Zesty / **Stretch** |
| Dec 2016 | **Dec 2021** | | **3.6.0** | 17.10 Artful - **18.04 Bionic** - 18.10 Cosmic |
| Oct 2016 | | | | | 16.10 Yakkety | 2.9.3 | 2.7.11 3.5.1 | 7.3 |
| Nov 2016 | *Sep 2017* | *2.11.0* | | | 17.04 Zesty / *Stretch* |
| Dec 2016 | **Dec 2021** | | **3.6.0** | | 17.10 Artful - **18.04 Bionic** - 18.10 Cosmic |
| Dec 2016 | | | | 7.4 | 17.04 Zesty / *Debian 9 Stretch* |
| Feb 2017 | *Sep 2017* | 2.12.0 |
| Apr 2017 | | | | 17.04 Zesty | 2.11.0 | 2.7.13 3.5.3 |
| Mar 2017 | | | | 7.5 | 17.10 Artful |
| Apr 2017 | | | | | 17.04 Zesty | 2.11.0 | 2.7.13 3.5.3 | 7.4 |
| May 2017 | *May 2018* | 2.13.0 |
| Jun 2017 | **Jun 2022** | | | **Debian 9 Stretch** | 2.11.0 | 2.7.13 3.5.3 |
| Aug 2017 | *Dec 2019* | 2.14.0 | | 17.10 Artful |
| Jun 2017 | *Jun 2022* | | | | *Debian 9 Stretch* | 2.11.0 | 2.7.13 3.5.3 | 7.4 |
| Aug 2017 | *Dec 2019* | 2.14.0 | | | 17.10 Artful |
| Oct 2017 | *Dec 2019* | 2.15.0 |
| Oct 2017 | | | | 17.10 Artful | 2.14.1 | 2.7.14 3.6.3 |
| Oct 2017 | | | | 7.6 | **18.04 Bionic** |
| Oct 2017 | | | | | 17.10 Artful | 2.14.1 | 2.7.14 3.6.3 | 7.5 |
| Jan 2018 | *Dec 2019* | 2.16.0 |
| Apr 2018 | *Dec 2019* | 2.17.0 | | **18.04 Bionic** |
| Apr 2018 | **Apr 2028** | | | **18.04 Bionic** | 2.17.0 | 2.7.15 3.6.5 |
| Jun 2018 | *Dec 2019* | 2.18.0 |
| Jun 2018 | **Jun 2023** | | 3.7.0 | 19.04 Disco - **20.04 Focal** / **Buster** |
| Sep 2018 | *Dec 2019* | 2.19.0 | | 18.10 Cosmic |
| Oct 2018 | | | | 18.10 Cosmic | 2.19.1 | 2.7.15 3.6.6 |
| Dec 2018 | *Dec 2019* | **2.20.0** | | 19.04 Disco / **Buster** |
| Feb 2019 | *Dec 2019* | 2.21.0 |
| Apr 2019 | | | | 19.04 Disco | 2.20.1 | 2.7.16 3.7.3 |
| Apr 2018 | *Mar 2021* | **2.17.0** | | | **18.04 Bionic** |
| Apr 2018 | | | | 7.7 | 18.10 Cosmic |
| Apr 2018 | **Apr 2028** | | | | **18.04 Bionic** | 2.17.0 | 2.7.15 3.6.5 | 7.6 |
| Jun 2018 | *Mar 2021* | 2.18.0 |
| Jun 2018 | **Jun 2023** | | 3.7.0 | | 19.04 Disco - **20.04 Focal** / **Buster** |
| Aug 2018 | | | | 7.8 |
| Sep 2018 | *Mar 2021* | 2.19.0 | | | 18.10 Cosmic |
| Oct 2018 | | | | 7.9 | 19.04 Disco / **Buster** |
| Oct 2018 | | | | | 18.10 Cosmic | 2.19.1 | 2.7.15 3.6.6 | 7.7 |
| Dec 2018 | *Mar 2021* | **2.20.0** | | | 19.04 Disco - 19.10 Eoan / **Buster** |
| Feb 2019 | *Mar 2021* | 2.21.0 |
| Apr 2019 | | | | 8.0 | 19.10 Eoan |
| Apr 2019 | | | | | 19.04 Disco | 2.20.1 | 2.7.16 3.7.3 | 7.9 |
| Jun 2019 | | 2.22.0 |
| Jul 2019 | **Jul 2024** | | | **Debian 10 Buster** | 2.20.1 | 2.7.16 3.7.3 |
| Aug 2019 | | 2.23.0 |
| Oct 2019 | **Oct 2024** | | 3.8.0 |
| Oct 2019 | | | | 19.10 Eoan | 2.20.1 | 2.7.17 3.7.5 |
| Nov 2019 | | 2.24.0 |
| Jan 2020 | | 2.25.0 | | **20.04 Focal** |
| Apr 2020 | **Apr 2030** | | | **20.04 Focal** | 2.25.0 | 2.7.17 3.7.5 |
| Oct 2020 | **Oct 2025** | | 3.9.0 | 21.04 Hirsute / **Bullseye** |
| Dec 2020 | | 2.30.0 | | 21.04 Hirsute / **Bullseye** |
| Apr 2021 | *Jan 2022* | | | 21.04 Hirsute | 2.30.2 | 2.7.18 3.9.4 |
| Aug 2021 | **Aug 2026** | | | **Debian 11 Bullseye** | 2.30.2 | 2.7.18 3.9.2 |
| **Date** | **EOL** | **[Git][rel-g]** | **[Python][rel-p]** | **[Ubuntu][rel-u] / [Debian][rel-d]** | **Git** | **Python** |
| Jul 2019 | **Jul 2024** | | | | **Debian 10 Buster** | 2.20.1 | 2.7.16 3.7.3 | 7.9 |
| Aug 2019 | *Mar 2021* | 2.23.0 |
| Oct 2019 | **Oct 2024** | | 3.8.0 | | **20.04 Focal** - 20.10 Groovy |
| Oct 2019 | | | | 8.1 |
| Oct 2019 | | | | | 19.10 Eoan | 2.20.1 | 2.7.17 3.7.5 | 8.0 |
| Nov 2019 | *Mar 2021* | 2.24.0 |
| Jan 2020 | *Mar 2021* | 2.25.0 | | | **20.04 Focal** |
| Feb 2020 | | | | 8.2 | **20.04 Focal** |
| Mar 2020 | *Mar 2021* | 2.26.0 |
| Apr 2020 | **Apr 2030** | | | | **20.04 Focal** | 2.25.1 | 2.7.17 3.8.2 | 8.2 |
| May 2020 | *Mar 2021* | 2.27.0 | | | 20.10 Groovy |
| May 2020 | | | | 8.3 |
| Jul 2020 | *Mar 2021* | 2.28.0 |
| Sep 2020 | | | | 8.4 | 21.04 Hirsute / **Bullseye** |
| Oct 2020 | *Mar 2021* | 2.29.0 |
| Oct 2020 | | | | | 20.10 Groovy | 2.27.0 | 2.7.18 3.8.6 | 8.3 |
| Oct 2020 | **Oct 2025** | | 3.9.0 | | 21.04 Hirsute / **Bullseye** |
| Dec 2020 | *Mar 2021* | 2.30.0 | | | 21.04 Hirsute / **Bullseye** |
| Mar 2021 | | 2.31.0 |
| Mar 2021 | | | | 8.5 |
| Apr 2021 | | | | 8.6 |
| Apr 2021 | *Jan 2022* | | | | 21.04 Hirsute | 2.30.2 | 2.7.18 3.9.4 | 8.4 |
| Jun 2021 | | 2.32.0 |
| Aug 2021 | | 2.33.0 |
| Aug 2021 | | | | 8.7 |
| Aug 2021 | **Aug 2026** | | | | **Debian 11 Bullseye** | 2.30.2 | 2.7.18 3.9.2 | 8.4 |
| **Date** | **EOL** | **[Git][rel-g]** | **[Python][rel-p]** | **[SSH][rel-o]** | **[Ubuntu][rel-u] / [Debian][rel-d]** | **Git** | **Python** | **SSH** |
[contact]: ../README.md#contact
[rel-d]: https://en.wikipedia.org/wiki/Debian_version_history
[rel-g]: https://en.wikipedia.org/wiki/Git#Releases
[rel-o]: https://www.openssh.com/releasenotes.html
[rel-p]: https://en.wikipedia.org/wiki/History_of_Python#Table_of_versions
[rel-u]: https://en.wikipedia.org/wiki/Ubuntu_version_history#Table_of_versions
[example announcement]: https://groups.google.com/d/topic/repo-discuss/UGBNismWo1M/discussion

45
fetch.py Normal file
View File

@ -0,0 +1,45 @@
# Copyright (C) 2021 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""This module contains functions used to fetch files from various sources."""
import subprocess
import sys
from urllib.parse import urlparse
from urllib.request import urlopen
def fetch_file(url, verbose=False):
"""Fetch a file from the specified source using the appropriate protocol.
Returns:
The contents of the file as bytes.
"""
scheme = urlparse(url).scheme
if scheme == 'gs':
cmd = ['gsutil', 'cat', url]
try:
result = subprocess.run(
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
check=True)
if result.stderr and verbose:
print('warning: non-fatal error running "gsutil": %s' % result.stderr,
file=sys.stderr)
return result.stdout
except subprocess.CalledProcessError as e:
print('fatal: error running "gsutil": %s' % e.stderr,
file=sys.stderr)
sys.exit(1)
with urlopen(url) as f:
return f.read()

View File

@ -104,6 +104,10 @@ class GitConfig(object):
os.path.dirname(self.file),
'.repo_' + os.path.basename(self.file) + '.json')
def ClearCache(self):
"""Clear the in-memory cache of config."""
self._cache_dict = None
def Has(self, name, include_defaults=True):
"""Return true if this configuration file has the key.
"""
@ -349,7 +353,7 @@ class GitConfig(object):
with open(self._json) as fd:
return json.load(fd)
except (IOError, ValueError):
platform_utils.remove(self._json)
platform_utils.remove(self._json, missing_ok=True)
return None
def _SaveJson(self, cache):
@ -357,8 +361,7 @@ class GitConfig(object):
with open(self._json, 'w') as fd:
json.dump(cache, fd, indent=2)
except (IOError, TypeError):
if os.path.exists(self._json):
platform_utils.remove(self._json)
platform_utils.remove(self._json, missing_ok=True)
def _ReadGit(self):
"""
@ -368,9 +371,10 @@ class GitConfig(object):
"""
c = {}
d = self._do('--null', '--list')
if d is None:
if not os.path.exists(self.file):
return c
d = self._do('--null', '--list')
for line in d.rstrip('\0').split('\0'):
if '\n' in line:
key, val = line.split('\n', 1)
@ -399,7 +403,7 @@ class GitConfig(object):
if p.Wait() == 0:
return p.stdout
else:
GitError('git config %s: %s' % (str(args), p.stderr))
raise GitError('git config %s: %s' % (str(args), p.stderr))
class RepoConfig(GitConfig):

View File

@ -90,7 +90,7 @@ class Superproject(object):
self._git_event_log = git_event_log
self._quiet = quiet
self._print_messages = print_messages
self._branch = self._GetBranch()
self._branch = manifest.branch
self._repodir = os.path.abspath(repodir)
self._superproject_dir = superproject_dir
self._superproject_path = os.path.join(self._repodir, superproject_dir)
@ -98,8 +98,12 @@ class Superproject(object):
_SUPERPROJECT_MANIFEST_NAME)
git_name = ''
if self._manifest.superproject:
remote_name = self._manifest.superproject['remote'].name
git_name = hashlib.md5(remote_name.encode('utf8')).hexdigest() + '-'
remote = self._manifest.superproject['remote']
git_name = hashlib.md5(remote.name.encode('utf8')).hexdigest() + '-'
self._branch = self._manifest.superproject['revision']
self._remote_url = remote.url
else:
self._remote_url = None
self._work_git_name = git_name + _SUPERPROJECT_GIT_NAME
self._work_git = os.path.join(self._superproject_path, self._work_git_name)
@ -113,30 +117,23 @@ class Superproject(object):
"""Returns the manifest path if the path exists or None."""
return self._manifest_path if os.path.exists(self._manifest_path) else None
def _GetBranch(self):
"""Returns the branch name for getting the approved manifest."""
p = self._manifest.manifestProject
b = p.GetBranch(p.CurrentBranch)
if not b:
return None
branch = b.merge
if branch and branch.startswith(R_HEADS):
branch = branch[len(R_HEADS):]
return branch
def _LogMessage(self, message):
"""Logs message to stderr and _git_event_log."""
if self._print_messages:
print(message, file=sys.stderr)
self._git_event_log.ErrorEvent(message, f'{message}')
def _LogMessagePrefix(self):
"""Returns the prefix string to be logged in each log message"""
return f'repo superproject branch: {self._branch} url: {self._remote_url}'
def _LogError(self, message):
"""Logs error message to stderr and _git_event_log."""
self._LogMessage(f'repo superproject error: {message}')
self._LogMessage(f'{self._LogMessagePrefix()} error: {message}')
def _LogWarning(self, message):
"""Logs warning message to stderr and _git_event_log."""
self._LogMessage(f'repo superproject warning: {message}')
self._LogMessage(f'{self._LogMessagePrefix()} warning: {message}')
def _Init(self):
"""Sets up a local Git repository to get a copy of a superproject.
@ -162,11 +159,8 @@ class Superproject(object):
return False
return True
def _Fetch(self, url):
"""Fetches a local copy of a superproject for the manifest based on url.
Args:
url: superproject's url.
def _Fetch(self):
"""Fetches a local copy of a superproject for the manifest based on |_remote_url|.
Returns:
True if fetch is successful, or False.
@ -177,7 +171,8 @@ class Superproject(object):
if not git_require((2, 28, 0)):
self._LogWarning('superproject requires a git version 2.28 or later')
return False
cmd = ['fetch', url, '--depth', '1', '--force', '--no-tags', '--filter', 'blob:none']
cmd = ['fetch', self._remote_url, '--depth', '1', '--force', '--no-tags',
'--filter', 'blob:none']
if self._branch:
cmd += [self._branch + ':' + self._branch]
p = GitCommand(None,
@ -234,15 +229,14 @@ class Superproject(object):
print('NOTICE: --use-superproject is in beta; report any issues to the '
'address described in `repo version`', file=sys.stderr)
should_exit = True
url = self._manifest.superproject['remote'].url
if not url:
if not self._remote_url:
self._LogWarning(f'superproject URL is not defined in manifest: '
f'{self._manifest.manifestFile}')
return SyncResult(False, should_exit)
if not self._Init():
return SyncResult(False, should_exit)
if not self._Fetch(url):
if not self._Fetch():
return SyncResult(False, should_exit)
if not self._quiet:
print('%s: Initial setup for superproject completed.' % self._work_git)
@ -260,7 +254,7 @@ class Superproject(object):
data = self._LsTree()
if not data:
self._LogWarning(f'warning: git ls-tree failed to return data for manifest: '
self._LogWarning(f'git ls-tree failed to return data for manifest: '
f'{self._manifest.manifestFile}')
return CommitIdsResult(None, True)
@ -366,57 +360,36 @@ def _UseSuperprojectFromConfiguration():
user_value = user_cfg.GetBoolean('repo.superprojectChoice')
if user_value is not None:
user_expiration = user_cfg.GetInt('repo.superprojectChoiceExpire')
if user_expiration is not None and (user_expiration <= 0 or user_expiration >= time_now):
if user_expiration is None or user_expiration <= 0 or user_expiration >= time_now:
# TODO(b/190688390) - Remove prompt when we are comfortable with the new
# default value.
print(('You are currently enrolled in Git submodules experiment '
'(go/android-submodules-quickstart). Use --no-use-superproject '
'to override.\n'), file=sys.stderr)
return user_value
if user_value:
print(('You are currently enrolled in Git submodules experiment '
'(go/android-submodules-quickstart). Use --no-use-superproject '
'to override.\n'), file=sys.stderr)
else:
print(('You are not currently enrolled in Git submodules experiment '
'(go/android-submodules-quickstart). Use --use-superproject '
'to override.\n'), file=sys.stderr)
return user_value
# We don't have an unexpired choice, ask for one.
system_cfg = RepoConfig.ForSystem()
system_value = system_cfg.GetBoolean('repo.superprojectChoice')
if system_value:
# The system configuration is proposing that we should enable the
# use of superproject. Present this to user for confirmation if we
# are on a TTY, or, when we are not on a TTY, accept the system
# default for this time only.
# use of superproject. Treat the user as enrolled for two weeks.
#
# TODO(b/190688390) - Remove prompt when we are comfortable with the new
# default value.
prompt = ('Repo can now use Git submodules (go/android-submodules-quickstart) '
'instead of manifests to represent the state of the Android '
'superproject, which results in faster syncs and better atomicity.\n\n')
if sys.stdout.isatty():
prompt += 'Would you like to opt in for two weeks (y/N)? '
response = input(prompt).lower()
time_choiceexpire = time_now + (86400 * 14)
if response in ('y', 'yes'):
userchoice = True
elif response in ('a', 'always'):
userchoice = True
time_choiceexpire = 0
elif response == 'never':
userchoice = False
time_choiceexpire = 0
elif response in ('n', 'no'):
userchoice = False
else:
# Unrecognized user response, assume the intention was no, but
# only for 2 hours instead of 2 weeks to balance between not
# being overly pushy while still retain the opportunity to
# enroll.
userchoice = False
time_choiceexpire = time_now + 7200
user_cfg.SetString('repo.superprojectChoiceExpire', str(time_choiceexpire))
user_cfg.SetBoolean('repo.superprojectChoice', userchoice)
return userchoice
else:
print('Accepting once since we are not on a TTY', file=sys.stderr)
return True
userchoice = True
time_choiceexpire = time_now + (86400 * 14)
user_cfg.SetString('repo.superprojectChoiceExpire', str(time_choiceexpire))
user_cfg.SetBoolean('repo.superprojectChoice', userchoice)
print('You are automatically enrolled in Git submodules experiment '
'(go/android-submodules-quickstart) for another two weeks.\n',
file=sys.stderr)
return True
# For all other cases, we would not use superproject by default.
return False
@ -437,4 +410,6 @@ def UseSuperproject(opt, manifest):
if client_value is not None:
return client_value
else:
if not manifest.superproject:
return False
return _UseSuperprojectFromConfiguration()

View File

@ -167,6 +167,26 @@ class EventLog(object):
repo_config = {k: v for k, v in config.items() if k.startswith('repo.')}
self.LogConfigEvents(repo_config, 'def_param')
def GetDataEventName(self, value):
"""Returns 'data-json' if the value is an array else returns 'data'."""
return 'data-json' if value[0] == '[' and value[-1] == ']' else 'data'
def LogDataConfigEvents(self, config, prefix):
"""Append a 'data' event for each config key/value in |config| to the current log.
For each keyX and valueX of the config, "key" field of the event is '|prefix|/keyX'
and the "value" of the "key" field is valueX.
Args:
config: Configuration dictionary.
prefix: Prefix for each key that is logged.
"""
for key, value in config.items():
event = self._CreateEventDict(self.GetDataEventName(value))
event['key'] = f'{prefix}/{key}'
event['value'] = value
self._log.append(event)
def ErrorEvent(self, msg, fmt):
"""Append a 'error' event to the current log."""
error_event = self._CreateEventDict('error')

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo gitc-init" "Repo Manual"
.TH REPO "1" "November 2021" "repo gitc-init" "Repo Manual"
.SH NAME
repo \- repo gitc-init - manual page for repo gitc-init
.SH SYNOPSIS
@ -41,6 +41,10 @@ platform group [auto|all|none|linux|darwin|...]
.TP
\fB\-\-submodules\fR
sync any submodules associated with the manifest repo
.TP
\fB\-\-standalone\-manifest\fR
download the manifest as a static file rather then
create a git checkout of the manifest repo
.SS Manifest (only) checkout options:
.TP
\fB\-\-current\-branch\fR
@ -92,7 +96,8 @@ filter for use with \fB\-\-partial\-clone\fR [default:
blob:none]
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects
use the manifest superproject to sync projects;
implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo init" "Repo Manual"
.TH REPO "1" "November 2021" "repo init" "Repo Manual"
.SH NAME
repo \- repo init - manual page for repo init
.SH SYNOPSIS
@ -41,6 +41,10 @@ platform group [auto|all|none|linux|darwin|...]
.TP
\fB\-\-submodules\fR
sync any submodules associated with the manifest repo
.TP
\fB\-\-standalone\-manifest\fR
download the manifest as a static file rather then
create a git checkout of the manifest repo
.SS Manifest (only) checkout options:
.TP
\fB\-c\fR, \fB\-\-current\-branch\fR
@ -92,7 +96,8 @@ filter for use with \fB\-\-partial\-clone\fR [default:
blob:none]
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects
use the manifest superproject to sync projects;
implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
@ -137,6 +142,12 @@ equivalent to using \fB\-b\fR HEAD.
The optional \fB\-m\fR argument can be used to specify an alternate manifest to be
used. If no manifest is specified, the manifest default.xml will be used.
.PP
If the \fB\-\-standalone\-manifest\fR argument is set, the manifest will be downloaded
directly from the specified \fB\-\-manifest\-url\fR as a static file (rather than setting
up a manifest git checkout). With \fB\-\-standalone\-manifest\fR, the manifest will be
fully static and will not be re\-downloaded during subsesquent `repo init` and
`repo sync` calls.
.PP
The \fB\-\-reference\fR option can be used to point to a directory that has the content
of a \fB\-\-mirror\fR sync. This will make the working directory use as much data as
possible from the local reference directory when fetching from the server. This

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo manifest" "Repo Manual"
.TH REPO "1" "November 2021" "repo manifest" "Repo Manual"
.SH NAME
repo \- repo manifest - manual page for repo manifest
.SH SYNOPSIS
@ -161,6 +161,7 @@ CDATA #IMPLIED>
<!ELEMENT extend\-project EMPTY>
<!ATTLIST extend\-project name CDATA #REQUIRED>
<!ATTLIST extend\-project path CDATA #IMPLIED>
<!ATTLIST extend\-project dest\-path CDATA #IMPLIED>
<!ATTLIST extend\-project groups CDATA #IMPLIED>
<!ATTLIST extend\-project revision CDATA #IMPLIED>
<!ATTLIST extend\-project remote CDATA #IMPLIED>
@ -174,8 +175,9 @@ CDATA #IMPLIED>
<!ATTLIST repo\-hooks enabled\-list CDATA #REQUIRED>
.IP
<!ELEMENT superproject EMPTY>
<!ATTLIST superproject name CDATA #REQUIRED>
<!ATTLIST superproject remote IDREF #IMPLIED>
<!ATTLIST superproject name CDATA #REQUIRED>
<!ATTLIST superproject remote IDREF #IMPLIED>
<!ATTLIST superproject revision CDATA #IMPLIED>
.IP
<!ELEMENT contactinfo EMPTY>
<!ATTLIST contactinfo bugurl CDATA #REQUIRED>
@ -385,6 +387,11 @@ original manifest.
Attribute `path`: If specified, limit the change to projects checked out at the
specified path, rather than all projects with the given name.
.PP
Attribute `dest\-path`: If specified, a path relative to the top directory of the
repo client where the Git working directory for this project should be placed.
This is used to move a project in the checkout by overriding the existing `path`
setting.
.PP
Attribute `groups`: List of additional groups to which this project belongs.
Same syntax as the corresponding element of `project`.
.PP
@ -477,6 +484,10 @@ project](#element\-project) for more information.
Attribute `remote`: Name of a previously defined remote element. If not supplied
the remote given by the default element is used.
.PP
Attribute `revision`: Name of the Git branch the manifest wants to track for
this superproject. If not supplied the revision given by the remote element is
used if applicable, else the default element is used.
.PP
Element contactinfo
.PP
*** *Note*: This is currently a WIP. ***

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo smartsync" "Repo Manual"
.TH REPO "1" "November 2021" "repo smartsync" "Repo Manual"
.SH NAME
repo \- repo smartsync - manual page for repo smartsync
.SH SYNOPSIS
@ -80,7 +80,8 @@ password to authenticate with the manifest server
fetch submodules from server
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects
use the manifest superproject to sync projects;
implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
@ -89,7 +90,7 @@ disable use of manifest superprojects
fetch tags
.TP
\fB\-\-no\-tags\fR
don't fetch tags
don't fetch tags (default)
.TP
\fB\-\-optimized\-fetch\fR
only fetch projects fixed to sha1 if revision does not
@ -100,6 +101,10 @@ number of times to retry fetches on transient errors
.TP
\fB\-\-prune\fR
delete refs that no longer exist on the remote
(default)
.TP
\fB\-\-no\-prune\fR
do not delete refs that no longer exist on the remote
.SS Logging options:
.TP
\fB\-v\fR, \fB\-\-verbose\fR

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo sync" "Repo Manual"
.TH REPO "1" "November 2021" "repo sync" "Repo Manual"
.SH NAME
repo \- repo sync - manual page for repo sync
.SH SYNOPSIS
@ -80,7 +80,8 @@ password to authenticate with the manifest server
fetch submodules from server
.TP
\fB\-\-use\-superproject\fR
use the manifest superproject to sync projects
use the manifest superproject to sync projects;
implies \fB\-c\fR
.TP
\fB\-\-no\-use\-superproject\fR
disable use of manifest superprojects
@ -89,7 +90,7 @@ disable use of manifest superprojects
fetch tags
.TP
\fB\-\-no\-tags\fR
don't fetch tags
don't fetch tags (default)
.TP
\fB\-\-optimized\-fetch\fR
only fetch projects fixed to sha1 if revision does not
@ -100,6 +101,10 @@ number of times to retry fetches on transient errors
.TP
\fB\-\-prune\fR
delete refs that no longer exist on the remote
(default)
.TP
\fB\-\-no\-prune\fR
do not delete refs that no longer exist on the remote
.TP
\fB\-s\fR, \fB\-\-smart\-sync\fR
smart sync using manifest from the latest known good

View File

@ -1,5 +1,5 @@
.\" DO NOT MODIFY THIS FILE! It was generated by help2man.
.TH REPO "1" "July 2021" "repo" "Repo Manual"
.TH REPO "1" "November 2021" "repo" "Repo Manual"
.SH NAME
repo \- repository management tool built on top of git
.SH SYNOPSIS
@ -43,7 +43,7 @@ filename of event log to append timeline to
.TP
\fB\-\-git\-trace2\-event\-log\fR=\fI\,GIT_TRACE2_EVENT_LOG\/\fR
directory to write git trace2 event log to
.SS "The complete list of recognized repo commands are:"
.SS "The complete list of recognized repo commands is:"
.TP
abandon
Permanently abandon a development branch

View File

@ -270,8 +270,7 @@ class XmlManifest(object):
self.Override(name)
# Old versions of repo would generate symlinks we need to clean up.
if os.path.lexists(self.manifestFile):
platform_utils.remove(self.manifestFile)
platform_utils.remove(self.manifestFile, missing_ok=True)
# This file is interpreted as if it existed inside the manifest repo.
# That allows us to use <include> with the relative file name.
with open(self.manifestFile, 'w') as fp:
@ -507,6 +506,9 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if not d.remote or remote.orig_name != remoteName:
remoteName = remote.orig_name
e.setAttribute('remote', remoteName)
revision = remote.revision or d.revisionExpr
if not revision or revision != self._superproject['revision']:
e.setAttribute('revision', self._superproject['revision'])
root.appendChild(e)
if self._contactinfo.bugurl != Wrapper().BUG_URL:
@ -852,6 +854,8 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
for subproject in project.subprojects:
recursively_add_projects(subproject)
repo_hooks_project = None
enabled_repo_hooks = None
for node in itertools.chain(*node_list):
if node.nodeName == 'project':
project = self._ParseProject(node)
@ -864,6 +868,7 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
'project: %s' % name)
path = node.getAttribute('path')
dest_path = node.getAttribute('dest-path')
groups = node.getAttribute('groups')
if groups:
groups = self._ParseList(groups)
@ -872,46 +877,37 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
if remote:
remote = self._get_remote(node)
named_projects = self._projects[name]
if dest_path and not path and len(named_projects) > 1:
raise ManifestParseError('extend-project cannot use dest-path when '
'matching multiple projects: %s' % name)
for p in self._projects[name]:
if path and p.relpath != path:
continue
if groups:
p.groups.extend(groups)
if revision:
p.revisionExpr = revision
if IsId(revision):
p.revisionId = revision
else:
p.revisionId = None
p.SetRevision(revision)
if remote:
p.remote = remote.ToRemoteSpec(name)
if node.nodeName == 'repo-hooks':
# Get the name of the project and the (space-separated) list of enabled.
repo_hooks_project = self._reqatt(node, 'in-project')
enabled_repo_hooks = self._ParseList(self._reqatt(node, 'enabled-list'))
if dest_path:
del self._paths[p.relpath]
relpath, worktree, gitdir, objdir, _ = self.GetProjectPaths(name, dest_path)
p.UpdatePaths(relpath, worktree, gitdir, objdir)
self._paths[p.relpath] = p
if node.nodeName == 'repo-hooks':
# Only one project can be the hooks project
if self._repo_hooks_project is not None:
if repo_hooks_project is not None:
raise ManifestParseError(
'duplicate repo-hooks in %s' %
(self.manifestFile))
# Store a reference to the Project.
try:
repo_hooks_projects = self._projects[repo_hooks_project]
except KeyError:
raise ManifestParseError(
'project %s not found for repo-hooks' %
(repo_hooks_project))
if len(repo_hooks_projects) != 1:
raise ManifestParseError(
'internal error parsing repo-hooks in %s' %
(self.manifestFile))
self._repo_hooks_project = repo_hooks_projects[0]
# Store the enabled hooks in the Project object.
self._repo_hooks_project.enabled_repo_hooks = enabled_repo_hooks
# Get the name of the project and the (space-separated) list of enabled.
repo_hooks_project = self._reqatt(node, 'in-project')
enabled_repo_hooks = self._ParseList(self._reqatt(node, 'enabled-list'))
if node.nodeName == 'superproject':
name = self._reqatt(node, 'name')
# There can only be one superproject.
@ -929,6 +925,13 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
raise ManifestParseError("no remote for superproject %s within %s" %
(name, self.manifestFile))
self._superproject['remote'] = remote.ToRemoteSpec(name)
revision = node.getAttribute('revision') or remote.revision
if not revision:
revision = self._default.revisionExpr
if not revision:
raise ManifestParseError('no revision for superproject %s within %s' %
(name, self.manifestFile))
self._superproject['revision'] = revision
if node.nodeName == 'contactinfo':
bugurl = self._reqatt(node, 'bugurl')
# This element can be repeated, later entries will clobber earlier ones.
@ -944,12 +947,30 @@ https://gerrit.googlesource.com/git-repo/+/HEAD/docs/manifest-format.md
# If the manifest removes the hooks project, treat it as if it deleted
# the repo-hooks element too.
if self._repo_hooks_project and (self._repo_hooks_project.name == name):
self._repo_hooks_project = None
if repo_hooks_project == name:
repo_hooks_project = None
elif not XmlBool(node, 'optional', False):
raise ManifestParseError('remove-project element specifies non-existent '
'project: %s' % name)
# Store repo hooks project information.
if repo_hooks_project:
# Store a reference to the Project.
try:
repo_hooks_projects = self._projects[repo_hooks_project]
except KeyError:
raise ManifestParseError(
'project %s not found for repo-hooks' %
(repo_hooks_project))
if len(repo_hooks_projects) != 1:
raise ManifestParseError(
'internal error parsing repo-hooks in %s' %
(self.manifestFile))
self._repo_hooks_project = repo_hooks_projects[0]
# Store the enabled hooks in the Project object.
self._repo_hooks_project.enabled_repo_hooks = enabled_repo_hooks
def _AddMetaProjectMirror(self, m):
name = None
m_url = m.GetRemote(m.remote.name).url

View File

@ -124,31 +124,30 @@ def rename(src, dst):
else:
raise
else:
os.rename(src, dst)
shutil.move(src, dst)
def remove(path):
def remove(path, missing_ok=False):
"""Remove (delete) the file path. This is a replacement for os.remove that
allows deleting read-only files on Windows, with support for long paths and
for deleting directory symbolic links.
Availability: Unix, Windows."""
if isWindows():
longpath = _makelongpath(path)
try:
os.remove(longpath)
except OSError as e:
if e.errno == errno.EACCES:
os.chmod(longpath, stat.S_IWRITE)
# Directory symbolic links must be deleted with 'rmdir'.
if islink(longpath) and isdir(longpath):
os.rmdir(longpath)
else:
os.remove(longpath)
longpath = _makelongpath(path) if isWindows() else path
try:
os.remove(longpath)
except OSError as e:
if e.errno == errno.EACCES:
os.chmod(longpath, stat.S_IWRITE)
# Directory symbolic links must be deleted with 'rmdir'.
if islink(longpath) and isdir(longpath):
os.rmdir(longpath)
else:
raise
else:
os.remove(path)
os.remove(longpath)
elif missing_ok and e.errno == errno.ENOENT:
pass
else:
raise
def walk(top, topdown=True, onerror=None, followlinks=False):

View File

@ -457,11 +457,7 @@ class RemoteSpec(object):
class Project(object):
# These objects can be shared between several working trees.
shareable_files = ['description', 'info']
shareable_dirs = ['hooks', 'objects', 'rr-cache', 'svn']
# These objects can only be used by a single working tree.
working_tree_files = ['config', 'packed-refs', 'shallow']
working_tree_dirs = ['logs', 'refs']
shareable_dirs = ['hooks', 'objects', 'rr-cache']
def __init__(self,
manifest,
@ -519,21 +515,8 @@ class Project(object):
self.client = self.manifest = manifest
self.name = name
self.remote = remote
self.gitdir = gitdir.replace('\\', '/')
self.objdir = objdir.replace('\\', '/')
if worktree:
self.worktree = os.path.normpath(worktree).replace('\\', '/')
else:
self.worktree = None
self.relpath = relpath
self.revisionExpr = revisionExpr
if revisionId is None \
and revisionExpr \
and IsId(revisionExpr):
self.revisionId = revisionExpr
else:
self.revisionId = revisionId
self.UpdatePaths(relpath, worktree, gitdir, objdir)
self.SetRevision(revisionExpr, revisionId=revisionId)
self.rebase = rebase
self.groups = groups
@ -556,16 +539,6 @@ class Project(object):
self.copyfiles = []
self.linkfiles = []
self.annotations = []
self.config = GitConfig.ForRepository(gitdir=self.gitdir,
defaults=self.client.globalConfig)
if self.worktree:
self.work_git = self._GitGetByExec(self, bare=False, gitdir=gitdir)
else:
self.work_git = None
self.bare_git = self._GitGetByExec(self, bare=True, gitdir=gitdir)
self.bare_ref = GitRefs(gitdir)
self.bare_objdir = self._GitGetByExec(self, bare=True, gitdir=objdir)
self.dest_branch = dest_branch
self.old_revision = old_revision
@ -573,6 +546,35 @@ class Project(object):
# project containing repo hooks.
self.enabled_repo_hooks = []
def SetRevision(self, revisionExpr, revisionId=None):
"""Set revisionId based on revision expression and id"""
self.revisionExpr = revisionExpr
if revisionId is None and revisionExpr and IsId(revisionExpr):
self.revisionId = self.revisionExpr
else:
self.revisionId = revisionId
def UpdatePaths(self, relpath, worktree, gitdir, objdir):
"""Update paths used by this project"""
self.gitdir = gitdir.replace('\\', '/')
self.objdir = objdir.replace('\\', '/')
if worktree:
self.worktree = os.path.normpath(worktree).replace('\\', '/')
else:
self.worktree = None
self.relpath = relpath
self.config = GitConfig.ForRepository(gitdir=self.gitdir,
defaults=self.manifest.globalConfig)
if self.worktree:
self.work_git = self._GitGetByExec(self, bare=False, gitdir=self.gitdir)
else:
self.work_git = None
self.bare_git = self._GitGetByExec(self, bare=True, gitdir=self.gitdir)
self.bare_ref = GitRefs(self.gitdir)
self.bare_objdir = self._GitGetByExec(self, bare=True, gitdir=self.objdir)
@property
def Derived(self):
return self.is_derived
@ -1182,10 +1184,8 @@ class Project(object):
self._InitMRef()
else:
self._InitMirrorHead()
try:
platform_utils.remove(os.path.join(self.gitdir, 'FETCH_HEAD'))
except OSError:
pass
platform_utils.remove(os.path.join(self.gitdir, 'FETCH_HEAD'),
missing_ok=True)
return True
def PostRepoUpgrade(self):
@ -2040,8 +2040,11 @@ class Project(object):
if current_branch_only:
if self.revisionExpr.startswith(R_TAGS):
# this is a tag and its sha1 value should never change
# This is a tag and its commit id should never change.
tag_name = self.revisionExpr[len(R_TAGS):]
elif self.upstream and self.upstream.startswith(R_TAGS):
# This is a tag and its commit id should never change.
tag_name = self.upstream[len(R_TAGS):]
if is_sha1 or tag_name is not None:
if self._CheckForImmutableRevision():
@ -2307,15 +2310,12 @@ class Project(object):
cmd.append('+refs/tags/*:refs/tags/*')
ok = GitCommand(self, cmd, bare=True).Wait() == 0
if os.path.exists(bundle_dst):
platform_utils.remove(bundle_dst)
if os.path.exists(bundle_tmp):
platform_utils.remove(bundle_tmp)
platform_utils.remove(bundle_dst, missing_ok=True)
platform_utils.remove(bundle_tmp, missing_ok=True)
return ok
def _FetchBundle(self, srcUrl, tmpPath, dstPath, quiet, verbose):
if os.path.exists(dstPath):
platform_utils.remove(dstPath)
platform_utils.remove(dstPath, missing_ok=True)
cmd = ['curl', '--fail', '--output', tmpPath, '--netrc', '--location']
if quiet:
@ -2439,7 +2439,7 @@ class Project(object):
if quiet:
cmd.append('-q')
if GitCommand(self, cmd).Wait() != 0:
raise GitError('%s submodule update --init --recursive %s ' % self.name)
raise GitError('%s submodule update --init --recursive ' % self.name)
def _Rebase(self, upstream, onto=None):
cmd = ['rebase']
@ -2465,6 +2465,8 @@ class Project(object):
os.makedirs(self.objdir)
self.bare_objdir.init()
self._UpdateHooks(quiet=quiet)
if self.use_git_worktrees:
# Enable per-worktree config file support if possible. This is more a
# nice-to-have feature for users rather than a hard requirement.
@ -2477,10 +2479,9 @@ class Project(object):
os.makedirs(self.gitdir)
if init_obj_dir or init_git_dir:
self._ReferenceGitDir(self.objdir, self.gitdir, share_refs=False,
copy_all=True)
self._ReferenceGitDir(self.objdir, self.gitdir, copy_all=True)
try:
self._CheckDirReference(self.objdir, self.gitdir, share_refs=False)
self._CheckDirReference(self.objdir, self.gitdir)
except GitError as e:
if force_sync:
print("Retrying clone after deleting %s" %
@ -2525,8 +2526,6 @@ class Project(object):
_lwrite(os.path.join(self.gitdir, 'objects/info/alternates'),
os.path.join(ref_dir, 'objects') + '\n')
self._UpdateHooks(quiet=quiet)
m = self.manifest.manifestProject.config
for key in ['user.name', 'user.email']:
if m.Has(key, include_defaults=False):
@ -2542,13 +2541,18 @@ class Project(object):
raise
def _UpdateHooks(self, quiet=False):
if os.path.exists(self.gitdir):
if os.path.exists(self.objdir):
self._InitHooks(quiet=quiet)
def _InitHooks(self, quiet=False):
hooks = platform_utils.realpath(self._gitdir_path('hooks'))
hooks = platform_utils.realpath(os.path.join(self.objdir, 'hooks'))
if not os.path.exists(hooks):
os.makedirs(hooks)
# Delete sample hooks. They're noise.
for hook in glob.glob(os.path.join(hooks, '*.sample')):
platform_utils.remove(hook, missing_ok=True)
for stock_hook in _ProjectHooks():
name = os.path.basename(stock_hook)
@ -2646,40 +2650,16 @@ class Project(object):
else:
active_git.symbolic_ref('-m', msg, ref, dst)
def _CheckDirReference(self, srcdir, destdir, share_refs):
def _CheckDirReference(self, srcdir, destdir):
# Git worktrees don't use symlinks to share at all.
if self.use_git_worktrees:
return
symlink_files = self.shareable_files[:]
symlink_dirs = self.shareable_dirs[:]
if share_refs:
symlink_files += self.working_tree_files
symlink_dirs += self.working_tree_dirs
to_symlink = symlink_files + symlink_dirs
for name in set(to_symlink):
for name in self.shareable_dirs:
# Try to self-heal a bit in simple cases.
dst_path = os.path.join(destdir, name)
src_path = os.path.join(srcdir, name)
if name in self.working_tree_dirs:
# If the dir is missing under .repo/projects/, create it.
if not os.path.exists(src_path):
os.makedirs(src_path)
elif name in self.working_tree_files:
# If it's a file under the checkout .git/ and the .repo/projects/ has
# nothing, move the file under the .repo/projects/ tree.
if not os.path.exists(src_path) and os.path.isfile(dst_path):
platform_utils.rename(dst_path, src_path)
# If the path exists under the .repo/projects/ and there's no symlink
# under the checkout .git/, recreate the symlink.
if name in self.working_tree_dirs or name in self.working_tree_files:
if os.path.exists(src_path) and not os.path.exists(dst_path):
platform_utils.symlink(
os.path.relpath(src_path, os.path.dirname(dst_path)), dst_path)
dst = platform_utils.realpath(dst_path)
if os.path.lexists(dst):
src = platform_utils.realpath(src_path)
@ -2692,23 +2672,17 @@ class Project(object):
' use `repo sync --force-sync {0}` to '
'proceed.'.format(self.relpath))
def _ReferenceGitDir(self, gitdir, dotgit, share_refs, copy_all):
def _ReferenceGitDir(self, gitdir, dotgit, copy_all):
"""Update |dotgit| to reference |gitdir|, using symlinks where possible.
Args:
gitdir: The bare git repository. Must already be initialized.
dotgit: The repository you would like to initialize.
share_refs: If true, |dotgit| will store its refs under |gitdir|.
Only one work tree can store refs under a given |gitdir|.
copy_all: If true, copy all remaining files from |gitdir| -> |dotgit|.
This saves you the effort of initializing |dotgit| yourself.
"""
symlink_files = self.shareable_files[:]
symlink_dirs = self.shareable_dirs[:]
if share_refs:
symlink_files += self.working_tree_files
symlink_dirs += self.working_tree_dirs
to_symlink = symlink_files + symlink_dirs
to_symlink = symlink_dirs
to_copy = []
if copy_all:
@ -2736,14 +2710,6 @@ class Project(object):
elif os.path.isfile(src):
shutil.copy(src, dst)
# If the source file doesn't exist, ensure the destination
# file doesn't either.
if name in symlink_files and not os.path.lexists(src):
try:
platform_utils.remove(dst)
except OSError:
pass
except OSError as e:
if e.errno == errno.EPERM:
raise DownloadError(self._get_symlink_error_message())
@ -2780,50 +2746,111 @@ class Project(object):
self._InitMRef()
def _InitWorkTree(self, force_sync=False, submodules=False):
realdotgit = os.path.join(self.worktree, '.git')
tmpdotgit = realdotgit + '.tmp'
init_dotgit = not os.path.exists(realdotgit)
if init_dotgit:
if self.use_git_worktrees:
"""Setup the worktree .git path.
This is the user-visible path like src/foo/.git/.
With non-git-worktrees, this will be a symlink to the .repo/projects/ path.
With git-worktrees, this will be a .git file using "gitdir: ..." syntax.
Older checkouts had .git/ directories. If we see that, migrate it.
This also handles changes in the manifest. Maybe this project was backed
by "foo/bar" on the server, but now it's "new/foo/bar". We have to update
the path we point to under .repo/projects/ to match.
"""
dotgit = os.path.join(self.worktree, '.git')
# If using an old layout style (a directory), migrate it.
if not platform_utils.islink(dotgit) and platform_utils.isdir(dotgit):
self._MigrateOldWorkTreeGitDir(dotgit)
init_dotgit = not os.path.exists(dotgit)
if self.use_git_worktrees:
if init_dotgit:
self._InitGitWorktree()
self._CopyAndLinkFiles()
return
dotgit = tmpdotgit
platform_utils.rmtree(tmpdotgit, ignore_errors=True)
os.makedirs(tmpdotgit)
self._ReferenceGitDir(self.gitdir, tmpdotgit, share_refs=True,
copy_all=False)
else:
dotgit = realdotgit
if not init_dotgit:
# See if the project has changed.
if platform_utils.realpath(self.gitdir) != platform_utils.realpath(dotgit):
platform_utils.remove(dotgit)
try:
self._CheckDirReference(self.gitdir, dotgit, share_refs=True)
except GitError as e:
if force_sync and not init_dotgit:
try:
platform_utils.rmtree(dotgit)
return self._InitWorkTree(force_sync=False, submodules=submodules)
except Exception:
raise e
raise e
if init_dotgit or not os.path.exists(dotgit):
os.makedirs(self.worktree, exist_ok=True)
platform_utils.symlink(os.path.relpath(self.gitdir, self.worktree), dotgit)
if init_dotgit:
_lwrite(os.path.join(tmpdotgit, HEAD), '%s\n' % self.GetRevisionId())
if init_dotgit:
_lwrite(os.path.join(dotgit, HEAD), '%s\n' % self.GetRevisionId())
# Now that the .git dir is fully set up, move it to its final home.
platform_utils.rename(tmpdotgit, realdotgit)
# Finish checking out the worktree.
cmd = ['read-tree', '--reset', '-u', '-v', HEAD]
if GitCommand(self, cmd).Wait() != 0:
raise GitError('Cannot initialize work tree for ' + self.name)
# Finish checking out the worktree.
cmd = ['read-tree', '--reset', '-u']
cmd.append('-v')
cmd.append(HEAD)
if GitCommand(self, cmd).Wait() != 0:
raise GitError('Cannot initialize work tree for ' + self.name)
if submodules:
self._SyncSubmodules(quiet=True)
self._CopyAndLinkFiles()
if submodules:
self._SyncSubmodules(quiet=True)
self._CopyAndLinkFiles()
@classmethod
def _MigrateOldWorkTreeGitDir(cls, dotgit):
"""Migrate the old worktree .git/ dir style to a symlink.
This logic specifically only uses state from |dotgit| to figure out where to
move content and not |self|. This way if the backing project also changed
places, we only do the .git/ dir to .git symlink migration here. The path
updates will happen independently.
"""
# Figure out where in .repo/projects/ it's pointing to.
if not os.path.islink(os.path.join(dotgit, 'refs')):
raise GitError(f'{dotgit}: unsupported checkout state')
gitdir = os.path.dirname(os.path.realpath(os.path.join(dotgit, 'refs')))
# Remove known symlink paths that exist in .repo/projects/.
KNOWN_LINKS = {
'config', 'description', 'hooks', 'info', 'logs', 'objects',
'packed-refs', 'refs', 'rr-cache', 'shallow', 'svn',
}
# Paths that we know will be in both, but are safe to clobber in .repo/projects/.
SAFE_TO_CLOBBER = {
'COMMIT_EDITMSG', 'FETCH_HEAD', 'HEAD', 'gitk.cache', 'index', 'ORIG_HEAD',
}
# First see if we'd succeed before starting the migration.
unknown_paths = []
for name in platform_utils.listdir(dotgit):
# Ignore all temporary/backup names. These are common with vim & emacs.
if name.endswith('~') or (name[0] == '#' and name[-1] == '#'):
continue
dotgit_path = os.path.join(dotgit, name)
if name in KNOWN_LINKS:
if not platform_utils.islink(dotgit_path):
unknown_paths.append(f'{dotgit_path}: should be a symlink')
else:
gitdir_path = os.path.join(gitdir, name)
if name not in SAFE_TO_CLOBBER and os.path.exists(gitdir_path):
unknown_paths.append(f'{dotgit_path}: unknown file; please file a bug')
if unknown_paths:
raise GitError('Aborting migration: ' + '\n'.join(unknown_paths))
# Now walk the paths and sync the .git/ to .repo/projects/.
for name in platform_utils.listdir(dotgit):
dotgit_path = os.path.join(dotgit, name)
# Ignore all temporary/backup names. These are common with vim & emacs.
if name.endswith('~') or (name[0] == '#' and name[-1] == '#'):
platform_utils.remove(dotgit_path)
elif name in KNOWN_LINKS:
platform_utils.remove(dotgit_path)
else:
gitdir_path = os.path.join(gitdir, name)
platform_utils.remove(gitdir_path, missing_ok=True)
platform_utils.rename(dotgit_path, gitdir_path)
# Now that the dir should be empty, clear it out, and symlink it over.
platform_utils.rmdir(dotgit)
platform_utils.symlink(os.path.relpath(gitdir, os.path.dirname(dotgit)), dotgit)
def _get_symlink_error_message(self):
if platform_utils.isWindows():
@ -2833,9 +2860,6 @@ class Project(object):
'for other options.')
return 'filesystem must support symlinks'
def _gitdir_path(self, path):
return platform_utils.realpath(os.path.join(self.gitdir, path))
def _revlist(self, *args, **kw):
a = []
a.extend(args)

View File

@ -20,6 +20,7 @@ This is intended to be run only by the official Repo release managers.
import argparse
import os
import re
import subprocess
import sys
@ -49,18 +50,37 @@ def check(opts):
util.run(opts, ['gpg', '--verify', f'{opts.launcher}.asc'])
def postmsg(opts):
def get_version(opts):
"""Get the version from |launcher|."""
# Make sure we don't search $PATH when signing the "repo" file in the cwd.
launcher = os.path.join('.', opts.launcher)
cmd = [launcher, '--version']
ret = util.run(opts, cmd, encoding='utf-8', stdout=subprocess.PIPE)
m = re.search(r'repo launcher version ([0-9.]+)', ret.stdout)
if not m:
sys.exit(f'{opts.launcher}: unable to detect repo version')
return m.group(1)
def postmsg(opts, version):
"""Helpful info to show at the end for release manager."""
print(f"""
Repo launcher bucket:
gs://git-repo-downloads/
To upload this launcher directly:
gsutil cp -a public-read {opts.launcher} {opts.launcher}.asc gs://git-repo-downloads/
You should first upload it with a specific version:
gsutil cp -a public-read {opts.launcher} gs://git-repo-downloads/repo-{version}
gsutil cp -a public-read {opts.launcher}.asc gs://git-repo-downloads/repo-{version}.asc
NB: You probably want to upload it with a specific version first, e.g.:
gsutil cp -a public-read {opts.launcher} gs://git-repo-downloads/repo-3.0
gsutil cp -a public-read {opts.launcher}.asc gs://git-repo-downloads/repo-3.0.asc
Then to make it the public default:
gsutil cp -a public-read gs://git-repo-downloads/repo-{version} gs://git-repo-downloads/repo
gsutil cp -a public-read gs://git-repo-downloads/repo-{version}.asc gs://git-repo-downloads/repo.asc
NB: If a rollback is necessary, the GS bucket archives old versions, and may be
accessed by specifying their unique id number.
gsutil ls -la gs://git-repo-downloads/repo gs://git-repo-downloads/repo.asc
gsutil cp -a public-read gs://git-repo-downloads/repo#<unique id> gs://git-repo-downloads/repo
gsutil cp -a public-read gs://git-repo-downloads/repo.asc#<unique id> gs://git-repo-downloads/repo.asc
""")
@ -103,9 +123,10 @@ def main(argv):
opts.keys = [util.KEYID_DSA, util.KEYID_RSA, util.KEYID_ECC]
util.import_release_key(opts)
version = get_version(opts)
sign(opts)
check(opts)
postmsg(opts)
postmsg(opts, version)
return 0

View File

@ -59,11 +59,11 @@ def main(argv):
version = RepoSourceVersion()
cmdlist = [['help2man', '-N', '-n', f'repo {cmd} - manual page for repo {cmd}',
'-S', f'repo {cmd}', '-m', 'Repo Manual', f'--version-string={version}',
'-o', MANDIR.joinpath(f'repo-{cmd}.1'), TOPDIR.joinpath('repo'),
'-o', MANDIR.joinpath(f'repo-{cmd}.1.tmp'), TOPDIR.joinpath('repo'),
'-h', f'help {cmd}'] for cmd in subcmds.all_commands]
cmdlist.append(['help2man', '-N', '-n', 'repository management tool built on top of git',
'-S', 'repo', '-m', 'Repo Manual', f'--version-string={version}',
'-o', MANDIR.joinpath('repo.1'), TOPDIR.joinpath('repo'),
'-o', MANDIR.joinpath('repo.1.tmp'), TOPDIR.joinpath('repo'),
'-h', '--help-all'])
with tempfile.TemporaryDirectory() as tempdir:
@ -80,11 +80,23 @@ def main(argv):
(r'^\.IP\n(.*:)\n', '.SS \g<1>\n'),
(r'^\.PP\nDescription', '.SH DETAILS'),
)
for path in MANDIR.glob('*.1'):
data = path.read_text()
for tmp_path in MANDIR.glob('*.1.tmp'):
path = tmp_path.parent / tmp_path.stem
old_data = path.read_text() if path.exists() else ''
data = tmp_path.read_text()
tmp_path.unlink()
for pattern, replacement in regex:
data = re.sub(pattern, replacement, data, flags=re.M)
path.write_text(data)
# If the only thing that changed was the date, don't refresh. This avoids
# a lot of noise when only one file actually updates.
old_data = re.sub(r'^(\.TH REPO "1" ")([^"]+)', r'\1', old_data, flags=re.M)
new_data = re.sub(r'^(\.TH REPO "1" ")([^"]+)', r'\1', data, flags=re.M)
if old_data != new_data:
path.write_text(data)
if __name__ == '__main__':
sys.exit(main(sys.argv[1:]))

8
repo
View File

@ -149,7 +149,7 @@ if not REPO_REV:
BUG_URL = 'https://bugs.chromium.org/p/gerrit/issues/entry?template=Repo+tool+issue'
# increment this whenever we make important changes to this script
VERSION = (2, 15)
VERSION = (2, 17)
# increment this if the MAINTAINER_KEYS block is modified
KEYRING_VERSION = (2, 3)
@ -312,6 +312,10 @@ def InitParser(parser, gitc_init=False):
metavar='PLATFORM')
group.add_option('--submodules', action='store_true',
help='sync any submodules associated with the manifest repo')
group.add_option('--standalone-manifest', action='store_true',
help='download the manifest as a static file '
'rather then create a git checkout of '
'the manifest repo')
# Options that only affect manifest project, and not any of the projects
# specified in the manifest itself.
@ -368,7 +372,7 @@ def InitParser(parser, gitc_init=False):
help='filter for use with --partial-clone '
'[default: %default]')
group.add_option('--use-superproject', action='store_true', default=None,
help='use the manifest superproject to sync projects')
help='use the manifest superproject to sync projects; implies -c')
group.add_option('--no-use-superproject', action='store_false',
dest='use_superproject',
help='disable use of manifest superprojects')

3
ssh.py
View File

@ -52,6 +52,9 @@ def version():
"""return ssh version as a tuple"""
try:
return _parse_ssh_version()
except FileNotFoundError:
print('fatal: ssh not installed', file=sys.stderr)
sys.exit(1)
except subprocess.CalledProcessError:
print('fatal: unable to detect ssh version', file=sys.stderr)
sys.exit(1)

View File

@ -53,7 +53,7 @@ Displays detailed usage information about a command.
self.PrintAllCommandsBody()
def PrintAllCommandsBody(self):
print('The complete list of recognized repo commands are:')
print('The complete list of recognized repo commands is:')
commandNames = list(sorted(all_commands))
self._PrintCommands(commandNames)
print("See 'repo help <command>' for more information on a "

View File

@ -24,6 +24,7 @@ from error import ManifestParseError
from project import SyncBuffer
from git_config import GitConfig
from git_command import git_require, MIN_GIT_VERSION_SOFT, MIN_GIT_VERSION_HARD
import fetch
import git_superproject
import platform_utils
from wrapper import Wrapper
@ -53,6 +54,12 @@ The optional -m argument can be used to specify an alternate manifest
to be used. If no manifest is specified, the manifest default.xml
will be used.
If the --standalone-manifest argument is set, the manifest will be downloaded
directly from the specified --manifest-url as a static file (rather than
setting up a manifest git checkout). With --standalone-manifest, the manifest
will be fully static and will not be re-downloaded during subsesquent
`repo init` and `repo sync` calls.
The --reference option can be used to point to a directory that
has the content of a --mirror sync. This will make the working
directory use as much data as possible from the local reference
@ -112,6 +119,27 @@ to update the working directory files.
m = self.manifest.manifestProject
is_new = not m.Exists
# If repo has already been initialized, we take -u with the absence of
# --standalone-manifest to mean "transition to a standard repo set up",
# which necessitates starting fresh.
# If --standalone-manifest is set, we always tear everything down and start
# anew.
if not is_new:
was_standalone_manifest = m.config.GetString('manifest.standalone')
if was_standalone_manifest and not opt.manifest_url:
print('fatal: repo was initialized with a standlone manifest, '
'cannot be re-initialized without --manifest-url/-u')
sys.exit(1)
if opt.standalone_manifest or (was_standalone_manifest and
opt.manifest_url):
m.config.ClearCache()
if m.gitdir and os.path.exists(m.gitdir):
platform_utils.rmtree(m.gitdir)
if m.worktree and os.path.exists(m.worktree):
platform_utils.rmtree(m.worktree)
is_new = not m.Exists
if is_new:
if not opt.manifest_url:
print('fatal: manifest url is required.', file=sys.stderr)
@ -136,6 +164,21 @@ to update the working directory files.
m._InitGitDir(mirror_git=mirrored_manifest_git)
# If standalone_manifest is set, mark the project as "standalone" -- we'll
# still do much of the manifests.git set up, but will avoid actual syncs to
# a remote.
standalone_manifest = False
if opt.standalone_manifest:
standalone_manifest = True
m.config.SetString('manifest.standalone', opt.manifest_url)
elif not opt.manifest_url and not opt.manifest_branch:
# If -u is set and --standalone-manifest is not, then we're not in
# standalone mode. Otherwise, use config to infer what we were in the last
# init.
standalone_manifest = bool(m.config.GetString('manifest.standalone'))
if not standalone_manifest:
m.config.SetString('manifest.standalone', None)
self._ConfigureDepth(opt)
# Set the remote URL before the remote branch as we might need it below.
@ -145,22 +188,23 @@ to update the working directory files.
r.ResetFetch()
r.Save()
if opt.manifest_branch:
if opt.manifest_branch == 'HEAD':
opt.manifest_branch = m.ResolveRemoteHead()
if opt.manifest_branch is None:
print('fatal: unable to resolve HEAD', file=sys.stderr)
sys.exit(1)
m.revisionExpr = opt.manifest_branch
else:
if is_new:
default_branch = m.ResolveRemoteHead()
if default_branch is None:
# If the remote doesn't have HEAD configured, default to master.
default_branch = 'refs/heads/master'
m.revisionExpr = default_branch
if not standalone_manifest:
if opt.manifest_branch:
if opt.manifest_branch == 'HEAD':
opt.manifest_branch = m.ResolveRemoteHead()
if opt.manifest_branch is None:
print('fatal: unable to resolve HEAD', file=sys.stderr)
sys.exit(1)
m.revisionExpr = opt.manifest_branch
else:
m.PreSync()
if is_new:
default_branch = m.ResolveRemoteHead()
if default_branch is None:
# If the remote doesn't have HEAD configured, default to master.
default_branch = 'refs/heads/master'
m.revisionExpr = default_branch
else:
m.PreSync()
groups = re.split(r'[,\s]+', opt.groups)
all_platforms = ['linux', 'darwin', 'windows']
@ -250,6 +294,16 @@ to update the working directory files.
if opt.use_superproject is not None:
m.config.SetBoolean('repo.superproject', opt.use_superproject)
if standalone_manifest:
if is_new:
manifest_name = 'default.xml'
manifest_data = fetch.fetch_file(opt.manifest_url, verbose=opt.verbose)
dest = os.path.join(m.worktree, manifest_name)
os.makedirs(os.path.dirname(dest), exist_ok=True)
with open(dest, 'wb') as f:
f.write(manifest_data)
return
if not m.Sync_NetworkHalf(is_new=is_new, quiet=opt.quiet, verbose=opt.verbose,
clone_bundle=opt.clone_bundle,
current_branch_only=opt.current_branch_only,
@ -423,8 +477,18 @@ to update the working directory files.
# Check this here, else manifest will be tagged "not new" and init won't be
# possible anymore without removing the .repo/manifests directory.
if opt.archive and opt.mirror:
self.OptionParser.error('--mirror and --archive cannot be used together.')
if opt.mirror:
if opt.archive:
self.OptionParser.error('--mirror and --archive cannot be used '
'together.')
if opt.use_superproject is not None:
self.OptionParser.error('--mirror and --use-superproject cannot be '
'used together.')
if opt.standalone_manifest and (opt.manifest_branch or
opt.manifest_name != 'default.xml'):
self.OptionParser.error('--manifest-branch and --manifest-name cannot'
' be used with --standalone-manifest.')
if args:
if opt.manifest_url:

View File

@ -12,7 +12,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import errno
import functools
import http.cookiejar as cookielib
import io
@ -235,24 +234,25 @@ later is required to fix a server side protocol bug.
dest='fetch_submodules', action='store_true',
help='fetch submodules from server')
p.add_option('--use-superproject', action='store_true',
help='use the manifest superproject to sync projects')
help='use the manifest superproject to sync projects; implies -c')
p.add_option('--no-use-superproject', action='store_false',
dest='use_superproject',
help='disable use of manifest superprojects')
p.add_option('--tags',
action='store_false',
p.add_option('--tags', action='store_true',
help='fetch tags')
p.add_option('--no-tags',
dest='tags', action='store_false',
help="don't fetch tags")
help="don't fetch tags (default)")
p.add_option('--optimized-fetch',
dest='optimized_fetch', action='store_true',
help='only fetch projects fixed to sha1 if revision does not exist locally')
p.add_option('--retry-fetches',
default=0, action='store', type='int',
help='number of times to retry fetches on transient errors')
p.add_option('--prune', dest='prune', action='store_true',
help='delete refs that no longer exist on the remote')
p.add_option('--prune', action='store_true',
help='delete refs that no longer exist on the remote (default)')
p.add_option('--no-prune', dest='prune', action='store_false',
help='do not delete refs that no longer exist on the remote')
if show_smart:
p.add_option('-s', '--smart-sync',
dest='smart_sync', action='store_true',
@ -448,8 +448,8 @@ later is required to fix a server side protocol bug.
else:
pm.update(inc=0, msg='warming up')
chunksize = 4
with multiprocessing.Pool(
jobs, initializer=self._FetchInitChild, initargs=(ssh_proxy,)) as pool:
with multiprocessing.Pool(jobs, initializer=self._FetchInitChild,
initargs=(ssh_proxy,)) as pool:
results = pool.imap_unordered(
functools.partial(self._FetchProjectList, opt),
projects_list,
@ -605,7 +605,7 @@ later is required to fix a server side protocol bug.
pm = Progress('Garbage collecting', len(projects), delay=False, quiet=opt.quiet)
pm.update(inc=0, msg='prescan')
gc_gitdirs = {}
tidy_dirs = {}
for project in projects:
# Make sure pruning never kicks in with shared projects.
if (not project.use_git_worktrees and
@ -622,17 +622,29 @@ later is required to fix a server side protocol bug.
% (project.relpath,),
file=sys.stderr)
project.config.SetString('gc.pruneExpire', 'never')
gc_gitdirs[project.gitdir] = project.bare_git
pm.update(inc=len(projects) - len(gc_gitdirs), msg='warming up')
project.config.SetString('gc.autoDetach', 'false')
# Only call git gc once per objdir, but call pack-refs for the remainder.
if project.objdir not in tidy_dirs:
tidy_dirs[project.objdir] = (
True, # Run a full gc.
project.bare_git,
)
elif project.gitdir not in tidy_dirs:
tidy_dirs[project.gitdir] = (
False, # Do not run a full gc; just run pack-refs.
project.bare_git,
)
cpu_count = os.cpu_count()
jobs = min(self.jobs, cpu_count)
if jobs < 2:
for bare_git in gc_gitdirs.values():
for (run_gc, bare_git) in tidy_dirs.values():
pm.update(msg=bare_git._project.name)
bare_git.gc('--auto')
if run_gc:
bare_git.gc('--auto')
else:
bare_git.pack_refs()
pm.end()
return
@ -641,11 +653,14 @@ later is required to fix a server side protocol bug.
threads = set()
sem = _threading.Semaphore(jobs)
def GC(bare_git):
def tidy_up(run_gc, bare_git):
pm.start(bare_git._project.name)
try:
try:
bare_git.gc('--auto', config=config)
if run_gc:
bare_git.gc('--auto', config=config)
else:
bare_git.pack_refs(config=config)
except GitError:
err_event.set()
except Exception:
@ -655,11 +670,11 @@ later is required to fix a server side protocol bug.
pm.finish(bare_git._project.name)
sem.release()
for bare_git in gc_gitdirs.values():
for (run_gc, bare_git) in tidy_dirs.values():
if err_event.is_set() and opt.fail_fast:
break
sem.acquire()
t = _threading.Thread(target=GC, args=(bare_git,))
t = _threading.Thread(target=tidy_up, args=(run_gc, bare_git,))
t.daemon = True
threads.add(t)
t.start()
@ -752,7 +767,7 @@ later is required to fix a server side protocol bug.
with open(copylinkfile_path, 'rb') as fp:
try:
old_copylinkfile_paths = json.load(fp)
except:
except Exception:
print('error: %s is not a json formatted file.' %
copylinkfile_path, file=sys.stderr)
platform_utils.remove(copylinkfile_path)
@ -767,13 +782,9 @@ later is required to fix a server side protocol bug.
set(new_copyfile_paths))
for need_remove_file in need_remove_files:
try:
platform_utils.remove(need_remove_file)
except OSError as e:
if e.errno == errno.ENOENT:
# Try to remove the updated copyfile or linkfile.
# So, if the file is not exist, nothing need to do.
pass
# Try to remove the updated copyfile or linkfile.
# So, if the file is not exist, nothing need to do.
platform_utils.remove(need_remove_file, missing_ok=True)
# Create copy-link-files.json, save dest path of "copyfile" and "linkfile".
with open(copylinkfile_path, 'w', encoding='utf-8') as fp:
@ -918,6 +929,9 @@ later is required to fix a server side protocol bug.
if None in [opt.manifest_server_username, opt.manifest_server_password]:
self.OptionParser.error('both -u and -p must be given')
if opt.prune is None:
opt.prune = True
def Execute(self, opt, args):
if opt.jobs:
self.jobs = opt.jobs
@ -958,18 +972,25 @@ later is required to fix a server side protocol bug.
file=sys.stderr)
mp = self.manifest.manifestProject
mp.PreSync()
is_standalone_manifest = mp.config.GetString('manifest.standalone')
if not is_standalone_manifest:
mp.PreSync()
if opt.repo_upgraded:
_PostRepoUpgrade(self.manifest, quiet=opt.quiet)
if not opt.mp_update:
print('Skipping update of local manifest project.')
else:
elif not is_standalone_manifest:
self._UpdateManifestProject(opt, mp, manifest_name)
load_local_manifests = not self.manifest.HasLocalManifests
use_superproject = git_superproject.UseSuperproject(opt, self.manifest)
if use_superproject and (self.manifest.IsMirror or self.manifest.IsArchive):
# Don't use superproject, because we have no working tree.
use_superproject = False
if opt.use_superproject is not None:
print('Defaulting to no-use-superproject because there is no working tree.')
superproject_logging_data = {
'superproject': use_superproject,
'haslocalmanifests': bool(self.manifest.HasLocalManifests),
@ -1092,19 +1113,28 @@ later is required to fix a server side protocol bug.
sys.exit(1)
# Log the previous sync analysis state from the config.
self.git_event_log.LogConfigEvents(mp.config.GetSyncAnalysisStateData(),
'previous_sync_state')
self.git_event_log.LogDataConfigEvents(mp.config.GetSyncAnalysisStateData(),
'previous_sync_state')
# Update and log with the new sync analysis state.
mp.config.UpdateSyncAnalysisState(opt, superproject_logging_data)
self.git_event_log.LogConfigEvents(mp.config.GetSyncAnalysisStateData(),
'current_sync_state')
self.git_event_log.LogDataConfigEvents(mp.config.GetSyncAnalysisStateData(),
'current_sync_state')
if not opt.quiet:
print('repo sync has finished successfully.')
def _PostRepoUpgrade(manifest, quiet=False):
# Link the docs for the internal .repo/ layout for people
link = os.path.join(manifest.repodir, 'internal-fs-layout.md')
if not platform_utils.islink(link):
target = os.path.join('repo', 'docs', 'internal-fs-layout.md')
try:
platform_utils.symlink(target, link)
except Exception:
pass
wrapper = Wrapper()
if wrapper.NeedSetupGnuPG():
wrapper.SetupGnuPG(quiet)
@ -1171,10 +1201,7 @@ class _FetchTimes(object):
with open(self._path) as f:
self._times = json.load(f)
except (IOError, ValueError):
try:
platform_utils.remove(self._path)
except OSError:
pass
platform_utils.remove(self._path, missing_ok=True)
self._times = {}
def Save(self):
@ -1192,10 +1219,7 @@ class _FetchTimes(object):
with open(self._path, 'w') as f:
json.dump(self._times, f, indent=2)
except (IOError, TypeError):
try:
platform_utils.remove(self._path)
except OSError:
pass
platform_utils.remove(self._path, missing_ok=True)
# This is a replacement for xmlrpc.client.Transport using urllib2
# and supporting persistent-http[s]. It cannot change hosts from

View File

@ -11,13 +11,3 @@
intk = 10k
intm = 10m
intg = 10g
[repo "syncstate.main"]
synctime = 2021-08-13T18:37:43.928600Z
version = 1
[repo "syncstate.sys"]
argv = ['/usr/bin/pytest-3']
[repo "syncstate.superproject"]
test = false
[repo "syncstate.options"]
verbose = true
mpupdate = false

View File

@ -104,25 +104,6 @@ class GitConfigReadOnlyTests(unittest.TestCase):
for key, value in TESTS:
self.assertEqual(value, self.config.GetInt('section.%s' % (key,)))
def test_GetSyncAnalysisStateData(self):
"""Test config entries with a sync state analysis data."""
superproject_logging_data = {}
superproject_logging_data['test'] = False
options = type('options', (object,), {})()
options.verbose = 'true'
options.mp_update = 'false'
TESTS = (
('superproject.test', 'false'),
('options.verbose', 'true'),
('options.mpupdate', 'false'),
('main.version', '1'),
)
self.config.UpdateSyncAnalysisState(options, superproject_logging_data)
sync_data = self.config.GetSyncAnalysisStateData()
for key, value in TESTS:
self.assertEqual(sync_data[f'{git_config.SYNC_STATE_PREFIX}{key}'], value)
self.assertTrue(sync_data[f'{git_config.SYNC_STATE_PREFIX}main.synctime'])
class GitConfigReadWriteTests(unittest.TestCase):
"""Read/write tests of the GitConfig class."""
@ -187,6 +168,25 @@ class GitConfigReadWriteTests(unittest.TestCase):
config = self.get_config()
self.assertIsNone(config.GetBoolean('foo.bar'))
def test_GetSyncAnalysisStateData(self):
"""Test config entries with a sync state analysis data."""
superproject_logging_data = {}
superproject_logging_data['test'] = False
options = type('options', (object,), {})()
options.verbose = 'true'
options.mp_update = 'false'
TESTS = (
('superproject.test', 'false'),
('options.verbose', 'true'),
('options.mpupdate', 'false'),
('main.version', '1'),
)
self.config.UpdateSyncAnalysisState(options, superproject_logging_data)
sync_data = self.config.GetSyncAnalysisStateData()
for key, value in TESTS:
self.assertEqual(sync_data[f'{git_config.SYNC_STATE_PREFIX}{key}'], value)
self.assertTrue(sync_data[f'{git_config.SYNC_STATE_PREFIX}main.synctime'])
if __name__ == '__main__':
unittest.main()

View File

@ -157,7 +157,7 @@ class SuperprojectTestCase(unittest.TestCase):
""")
self._superproject = git_superproject.Superproject(manifest, self.repodir,
self.git_event_log)
with mock.patch.object(self._superproject, '_GetBranch', return_value='junk'):
with mock.patch.object(self._superproject, '_branch', 'junk'):
sync_result = self._superproject.Sync()
self.assertFalse(sync_result.success)
self.assertTrue(sync_result.fatal)

View File

@ -42,7 +42,7 @@ class EventLogTestCase(unittest.TestCase):
self._event_log_module = git_trace2_event_log.EventLog(env=env)
self._log_data = None
def verifyCommonKeys(self, log_entry, expected_event_name, full_sid=True):
def verifyCommonKeys(self, log_entry, expected_event_name=None, full_sid=True):
"""Helper function to verify common event log keys."""
self.assertIn('event', log_entry)
self.assertIn('sid', log_entry)
@ -50,7 +50,8 @@ class EventLogTestCase(unittest.TestCase):
self.assertIn('time', log_entry)
# Do basic data format validation.
self.assertEqual(expected_event_name, log_entry['event'])
if expected_event_name:
self.assertEqual(expected_event_name, log_entry['event'])
if full_sid:
self.assertRegex(log_entry['sid'], self.FULL_SID_REGEX)
else:
@ -65,6 +66,13 @@ class EventLogTestCase(unittest.TestCase):
log_data.append(json.loads(line))
return log_data
def remove_prefix(self, s, prefix):
"""Return a copy string after removing |prefix| from |s|, if present or the original string."""
if s.startswith(prefix):
return s[len(prefix):]
else:
return s
def test_initial_state_with_parent_sid(self):
"""Test initial state when 'GIT_TRACE2_PARENT_SID' is set by parent."""
self.assertRegex(self._event_log_module.full_sid, self.FULL_SID_REGEX)
@ -234,6 +242,42 @@ class EventLogTestCase(unittest.TestCase):
self.assertEqual(len(self._log_data), 1)
self.verifyCommonKeys(self._log_data[0], expected_event_name='version')
def test_data_event_config(self):
"""Test 'data' event data outputs all config keys.
Expected event log:
<version event>
<data event>
<data event>
"""
config = {
'git.foo': 'bar',
'repo.partialclone': 'false',
'repo.syncstate.superproject.hassuperprojecttag': 'true',
'repo.syncstate.superproject.sys.argv': ['--', 'sync', 'protobuf'],
}
prefix_value = 'prefix'
self._event_log_module.LogDataConfigEvents(config, prefix_value)
with tempfile.TemporaryDirectory(prefix='event_log_tests') as tempdir:
log_path = self._event_log_module.Write(path=tempdir)
self._log_data = self.readLog(log_path)
self.assertEqual(len(self._log_data), 5)
data_events = self._log_data[1:]
self.verifyCommonKeys(self._log_data[0], expected_event_name='version')
for event in data_events:
self.verifyCommonKeys(event)
# Check for 'data' event specific fields.
self.assertIn('key', event)
self.assertIn('value', event)
key = event['key']
key = self.remove_prefix(key, f'{prefix_value}/')
value = event['value']
self.assertEqual(self._event_log_module.GetDataEventName(value), event['event'])
self.assertTrue(key in config and value == config[key])
def test_error_event(self):
"""Test and validate 'error' event data is valid.

View File

@ -261,6 +261,19 @@ class XmlManifestTests(ManifestParseTestCase):
<project name="repohooks" path="src/repohooks"/>
<repo-hooks in-project="repohooks" enabled-list="a, b"/>
</manifest>
""")
self.assertEqual(manifest.repo_hooks_project.name, 'repohooks')
self.assertEqual(manifest.repo_hooks_project.enabled_repo_hooks, ['a', 'b'])
def test_repo_hooks_unordered(self):
"""Check repo-hooks settings work even if the project def comes second."""
manifest = self.getXmlManifest("""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/main" />
<repo-hooks in-project="repohooks" enabled-list="a, b"/>
<project name="repohooks" path="src/repohooks"/>
</manifest>
""")
self.assertEqual(manifest.repo_hooks_project.name, 'repohooks')
self.assertEqual(manifest.repo_hooks_project.enabled_repo_hooks, ['a', 'b'])
@ -559,6 +572,7 @@ class SuperProjectElementTests(ManifestParseTestCase):
self.assertEqual(manifest.superproject['name'], 'superproject')
self.assertEqual(manifest.superproject['remote'].name, 'test-remote')
self.assertEqual(manifest.superproject['remote'].url, 'http://localhost/superproject')
self.assertEqual(manifest.superproject['revision'], 'refs/heads/main')
self.assertEqual(
sort_attributes(manifest.ToXml().toxml()),
'<?xml version="1.0" ?><manifest>'
@ -567,6 +581,72 @@ class SuperProjectElementTests(ManifestParseTestCase):
'<superproject name="superproject"/>'
'</manifest>')
def test_superproject_revision(self):
"""Check superproject settings with a different revision attribute"""
self.maxDiff = None
manifest = self.getXmlManifest("""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/main" />
<superproject name="superproject" revision="refs/heads/stable" />
</manifest>
""")
self.assertEqual(manifest.superproject['name'], 'superproject')
self.assertEqual(manifest.superproject['remote'].name, 'test-remote')
self.assertEqual(manifest.superproject['remote'].url, 'http://localhost/superproject')
self.assertEqual(manifest.superproject['revision'], 'refs/heads/stable')
self.assertEqual(
sort_attributes(manifest.ToXml().toxml()),
'<?xml version="1.0" ?><manifest>'
'<remote fetch="http://localhost" name="test-remote"/>'
'<default remote="test-remote" revision="refs/heads/main"/>'
'<superproject name="superproject" revision="refs/heads/stable"/>'
'</manifest>')
def test_superproject_revision_default_negative(self):
"""Check superproject settings with a same revision attribute"""
self.maxDiff = None
manifest = self.getXmlManifest("""
<manifest>
<remote name="test-remote" fetch="http://localhost" />
<default remote="test-remote" revision="refs/heads/stable" />
<superproject name="superproject" revision="refs/heads/stable" />
</manifest>
""")
self.assertEqual(manifest.superproject['name'], 'superproject')
self.assertEqual(manifest.superproject['remote'].name, 'test-remote')
self.assertEqual(manifest.superproject['remote'].url, 'http://localhost/superproject')
self.assertEqual(manifest.superproject['revision'], 'refs/heads/stable')
self.assertEqual(
sort_attributes(manifest.ToXml().toxml()),
'<?xml version="1.0" ?><manifest>'
'<remote fetch="http://localhost" name="test-remote"/>'
'<default remote="test-remote" revision="refs/heads/stable"/>'
'<superproject name="superproject"/>'
'</manifest>')
def test_superproject_revision_remote(self):
"""Check superproject settings with a same revision attribute"""
self.maxDiff = None
manifest = self.getXmlManifest("""
<manifest>
<remote name="test-remote" fetch="http://localhost" revision="refs/heads/main" />
<default remote="test-remote" />
<superproject name="superproject" revision="refs/heads/stable" />
</manifest>
""")
self.assertEqual(manifest.superproject['name'], 'superproject')
self.assertEqual(manifest.superproject['remote'].name, 'test-remote')
self.assertEqual(manifest.superproject['remote'].url, 'http://localhost/superproject')
self.assertEqual(manifest.superproject['revision'], 'refs/heads/stable')
self.assertEqual(
sort_attributes(manifest.ToXml().toxml()),
'<?xml version="1.0" ?><manifest>'
'<remote fetch="http://localhost" name="test-remote" revision="refs/heads/main"/>'
'<default remote="test-remote"/>'
'<superproject name="superproject" revision="refs/heads/stable"/>'
'</manifest>')
def test_remote(self):
"""Check superproject settings with a remote."""
manifest = self.getXmlManifest("""
@ -580,6 +660,7 @@ class SuperProjectElementTests(ManifestParseTestCase):
self.assertEqual(manifest.superproject['name'], 'platform/superproject')
self.assertEqual(manifest.superproject['remote'].name, 'superproject-remote')
self.assertEqual(manifest.superproject['remote'].url, 'http://localhost/platform/superproject')
self.assertEqual(manifest.superproject['revision'], 'refs/heads/main')
self.assertEqual(
sort_attributes(manifest.ToXml().toxml()),
'<?xml version="1.0" ?><manifest>'
@ -600,6 +681,7 @@ class SuperProjectElementTests(ManifestParseTestCase):
""")
self.assertEqual(manifest.superproject['name'], 'superproject')
self.assertEqual(manifest.superproject['remote'].name, 'default-remote')
self.assertEqual(manifest.superproject['revision'], 'refs/heads/main')
self.assertEqual(
sort_attributes(manifest.ToXml().toxml()),
'<?xml version="1.0" ?><manifest>'
@ -715,3 +797,49 @@ class RemoveProjectElementTests(ManifestParseTestCase):
</manifest>
""")
self.assertEqual(manifest.projects, [])
class ExtendProjectElementTests(ManifestParseTestCase):
"""Tests for <extend-project>."""
def test_extend_project_dest_path_single_match(self):
manifest = self.getXmlManifest("""
<manifest>
<remote name="default-remote" fetch="http://localhost" />
<default remote="default-remote" revision="refs/heads/main" />
<project name="myproject" />
<extend-project name="myproject" dest-path="bar" />
</manifest>
""")
self.assertEqual(len(manifest.projects), 1)
self.assertEqual(manifest.projects[0].relpath, 'bar')
def test_extend_project_dest_path_multi_match(self):
with self.assertRaises(manifest_xml.ManifestParseError):
manifest = self.getXmlManifest("""
<manifest>
<remote name="default-remote" fetch="http://localhost" />
<default remote="default-remote" revision="refs/heads/main" />
<project name="myproject" path="x" />
<project name="myproject" path="y" />
<extend-project name="myproject" dest-path="bar" />
</manifest>
""")
manifest.projects
def test_extend_project_dest_path_multi_match_path_specified(self):
manifest = self.getXmlManifest("""
<manifest>
<remote name="default-remote" fetch="http://localhost" />
<default remote="default-remote" revision="refs/heads/main" />
<project name="myproject" path="x" />
<project name="myproject" path="y" />
<extend-project name="myproject" path="x" dest-path="bar" />
</manifest>
""")
self.assertEqual(len(manifest.projects), 2)
if manifest.projects[0].relpath == 'y':
self.assertEqual(manifest.projects[1].relpath, 'bar')
else:
self.assertEqual(manifest.projects[0].relpath, 'bar')
self.assertEqual(manifest.projects[1].relpath, 'y')

View File

@ -0,0 +1,50 @@
# Copyright 2021 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the platform_utils.py module."""
import os
import tempfile
import unittest
import platform_utils
class RemoveTests(unittest.TestCase):
"""Check remove() helper."""
def testMissingOk(self):
"""Check missing_ok handling."""
with tempfile.TemporaryDirectory() as tmpdir:
path = os.path.join(tmpdir, 'test')
# Should not fail.
platform_utils.remove(path, missing_ok=True)
# Should fail.
self.assertRaises(OSError, platform_utils.remove, path)
self.assertRaises(OSError, platform_utils.remove, path, missing_ok=False)
# Should not fail if it exists.
open(path, 'w').close()
platform_utils.remove(path, missing_ok=True)
self.assertFalse(os.path.exists(path))
open(path, 'w').close()
platform_utils.remove(path)
self.assertFalse(os.path.exists(path))
open(path, 'w').close()
platform_utils.remove(path, missing_ok=False)
self.assertFalse(os.path.exists(path))

View File

@ -16,6 +16,7 @@
import contextlib
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
@ -335,3 +336,76 @@ class LinkFile(CopyLinkTestCase):
platform_utils.symlink(self.tempdir, dest)
lf._Link()
self.assertEqual(os.path.join('git-project', 'foo.txt'), os.readlink(dest))
class MigrateWorkTreeTests(unittest.TestCase):
"""Check _MigrateOldWorkTreeGitDir handling."""
_SYMLINKS = {
'config', 'description', 'hooks', 'info', 'logs', 'objects',
'packed-refs', 'refs', 'rr-cache', 'shallow', 'svn',
}
_FILES = {
'COMMIT_EDITMSG', 'FETCH_HEAD', 'HEAD', 'index', 'ORIG_HEAD',
'unknown-file-should-be-migrated',
}
_CLEAN_FILES = {
'a-vim-temp-file~', '#an-emacs-temp-file#',
}
@classmethod
@contextlib.contextmanager
def _simple_layout(cls):
"""Create a simple repo client checkout to test against."""
with tempfile.TemporaryDirectory() as tempdir:
tempdir = Path(tempdir)
gitdir = tempdir / '.repo/projects/src/test.git'
gitdir.mkdir(parents=True)
cmd = ['git', 'init', '--bare', str(gitdir)]
subprocess.check_call(cmd)
dotgit = tempdir / 'src/test/.git'
dotgit.mkdir(parents=True)
for name in cls._SYMLINKS:
(dotgit / name).symlink_to(f'../../../.repo/projects/src/test.git/{name}')
for name in cls._FILES | cls._CLEAN_FILES:
(dotgit / name).write_text(name)
yield tempdir
def test_standard(self):
"""Migrate a standard checkout that we expect."""
with self._simple_layout() as tempdir:
dotgit = tempdir / 'src/test/.git'
project.Project._MigrateOldWorkTreeGitDir(str(dotgit))
# Make sure the dir was transformed into a symlink.
self.assertTrue(dotgit.is_symlink())
self.assertEqual(str(dotgit.readlink()), '../../.repo/projects/src/test.git')
# Make sure files were moved over.
gitdir = tempdir / '.repo/projects/src/test.git'
for name in self._FILES:
self.assertEqual(name, (gitdir / name).read_text())
# Make sure files were removed.
for name in self._CLEAN_FILES:
self.assertFalse((gitdir / name).exists())
def test_unknown(self):
"""A checkout with unknown files should abort."""
with self._simple_layout() as tempdir:
dotgit = tempdir / 'src/test/.git'
(tempdir / '.repo/projects/src/test.git/random-file').write_text('one')
(dotgit / 'random-file').write_text('two')
with self.assertRaises(error.GitError):
project.Project._MigrateOldWorkTreeGitDir(str(dotgit))
# Make sure no content was actually changed.
self.assertTrue(dotgit.is_dir())
for name in self._FILES:
self.assertTrue((dotgit / name).is_file())
for name in self._CLEAN_FILES:
self.assertTrue((dotgit / name).is_file())
for name in self._SYMLINKS:
self.assertTrue((dotgit / name).is_symlink())