Verify all the .git/ paths will be handled by the migration logic before
starting the migration. This way we still abort & log an error, but the
user gets to see it before we put the tree into a state that they have to
manually recover. Also add a few more known-safe-to-clobber paths.
Bug: https://crbug.com/gerrit/15273
Change-Id: If49d69b341bc960ddcafa30da333fb5ec7145b51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327557
Reviewed-by: Colin Cross <ccross@android.com>
Tested-by: Mike Frysinger <vapier@google.com>
This reverts commit d53cb9549a. As long as
repo's reference docs treat this feature as a work in progress and don't
cover it well enough to allow all repo maintainers to easily support it,
it is inconsistent to report to users that it is no longer in beta.
Thanks for vapier@google.com for noticing.
https://crbug.com/gerrit/15527 tracks the required documentation changes
before we'd be ready to roll forward again.
Change-Id: Ic9bd951cfb3c1abf6e1bfa30dfe4afa1c9b7bec6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327337
Reviewed-by: Jonathan Nieder <jrn@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jonathan Nieder <jrn@google.com>
Unsharing this directory shouldn't be a problem. The current repo code
treated it as a file, and while that's actually incorrect, files & dirs
are basically treated the same, so it's practically the same.
Let's enumerate each subpath since there aren't that many.
info/refs:
Only used when the project is exported over git dumb transports (i.e.
a http:// server). Repo never does this, and it's extremely unlikely
any user has ever done this. Plus, this proposal talks about unsharing
project refs, so this file should get unshared too.
info/grafts:
A user-configurable file that repo never touches. Might be useful to
share across projects, but probably rarely (if ever) used by developers,
and forcing them to configure it for each project isn't that big of a
deal.
info/exclude:
info/attributes:
User-configurable files that repo never touches. Doesn't seem like
most users ever touch these, and if they do, having them do it for
each shared project isn't a big deal.
info/sparse-checkout:
Repo doesn't use sparse checkouts, and it's extremely unlikely to even
work if a user tried doing something themselves.
Bug: https://crbug.com/gerrit/15460
Change-Id: I53e44d73a6d7a92da615b46600d8ea51cb46e3ac
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327519
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Nothing uses this path. It’s only for exporting git dirs e.g. for
online gitweb use which probably no one does. It is not the same
description file as exists on servers we cloned from. Leaving it
as the default plain text file will simplify code.
We don't undo any existing symlinks if they exist since repo does
not care about them, and their existence doesn't hurt.
Bug: https://crbug.com/gerrit/15460
Change-Id: Ic34fe7c3cfb8f6da844de5be30158f59382b1cc8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327518
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This path only matters to users of `git svn` who manually run it in
local projects after they get a full repo client checkout. With svn
usage falling in general, and with the fact that the source checkout
now symlinks its .git/ state to the internal projects/ path, we don't
need to manage this anymore.
It means the path won't be shared among multiple local projects that
have the same remote, but so it goes. It was an optimization only,
not functionality required for correctness. We want to simplify the
internals to stop messing with git state, and this particular path
doesn't seem worth the effort to maintain.
We don't undo any existing svn symlinks if they exist since repo does
not care about them, and their existence doesn't hurt anything.
Bug: https://crbug.com/gerrit/15460
Change-Id: Ie8496b275bcc589771aa9f4ee874ed2ee6d5241d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327517
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Now that we symlink worktree .git/ paths to .repo/projects/, we never
set share_refs=True anywhere, which means all of this logic is dead
code. Throw it all away. Do it as a separate commit to make the
parent commit easier to review.
Bug: https://crbug.com/gerrit/15273
Change-Id: If496d39029d3d3bd523ba24c603ce47a63ad9b51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/326817
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jack Neus <jackneus@google.com>
+ Display 'Defaulting to no-use-superproject because there is no working tree.'
message if --use-superproject option is used and we are not using
superproject because manifest is either a mirror or is an archive.
Tested the code with the following commands.
$ ./run_tests -v
Tested the sync code by using repo_dev alias and pointing to this CL.
$ repo init -u https://android.googlesource.com/mirror/manifest --mirror
$ repo_dev sync
Receiving objects: 100% (3/3), done.eiving objects: 33% (1/3)
$ repo_dev sync --use-superproject
Defaulting to no-use-superproject because there is no working tree.
Fetching: 0% (0/2158) warming up
Bug: https://crbug.com/gerrit/15368
Change-Id: I16b87ee9623315dbc3100b612b1decdaab7ac1dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/325797
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
Historically we created a .git/ subdir in each source checkout and
symlinked individual files to the .repo/projects/ paths. This layer
of indirection isn't actually needed: the .repo/projects/ paths are
guaranteed to only ever have a 1-to-1 mapping with the actual git
checkout. So we don't need to worry about having files in .git/ be
isolated.
To that end, change how we manage the actual project checkouts from
a dir full of symlinks (and a few files) to a symlink to the internal
.repo/projects/ dir. This makes the code simpler & faster.
The directory structure we have today is:
.repo/
project-objects/chromiumos/third_party/kernel.git/
<paths omitted as not relevant to this change>
projects/src/third_party/kernel/
v3.8.git/
config
description -> …/project-objects/…/config
FETCH_HEAD
HEAD
hooks/ -> …/project-objects/…/hooks/
info/ -> …/project-objects/…/info/
logs/
objects/ -> …/project-objects/…/objects/
packed-refs
refs/
rr-cache/ -> …/project-objects/…/rr-cache/
src/third_party/kernel/
v3.8/
.git/
config -> …/projects/…/v3.8.git/config
description -> …/project-objects/…/v3.8.git/description
HEAD
hooks/ -> …/project-objects/…/v3.8.git/hooks/
index
info/ -> …/project-objects/…/v3.8.git/info/
logs/ -> …/projects/…/v3.8.git/logs/
objects/ -> …/project-objects/…/v3.8.git/objects/
packed-refs -> …/projects/…/v3.8.git/packed-refs
refs/ -> …/projects/…/v3.8.git/refs/
rr-cache/ -> …/project-objects/…/v3.8.git/rr-cache/
The directory structure we have after this commit:
.repo/
<nothing changes>
src/third_party/kernel/
v3.8/
.git -> …/projects/…/v3.8.git
Bug: https://crbug.com/gerrit/15273
Change-Id: I9dd8def23fbfb2f4cb209a93f8b1b2b24002a444
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323695
Reviewed-by: Mike Nichols <mikenichols@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
init.py
+ Similar to opt.archive, gave an error if --mirror option is
used with --use-superproject.
sync.py
+ Defaulted to --no-use-superproject if manifest is a mirror or
archive (similar to error at line# 1067).
Tested:
+ run_tests
+ flake8 (will fix known errors in another CL).
$ repo_dev init -u sso://googleplex-android.git.corp.google.com/platform/manifest --use-superproject --mirror
Usage: repo init [options] [manifest url]
main.py: error: --mirror and --use-superproject cannot be used together.
+ repo init and repo sync with --mirror and without --mirror
options.
$ repo_dev init -u https://android.googlesource.com/platform/manifest
$ repo_dev sync
...superproject.git: Initial setup for superproject completed.
+ With --mirror option, verfied there are no exceptions in git_superproject.py
Bug: [google internal] b/206537893
Change-Id: I059f20e76f0ab36f0587f29779bb53ede4663bd4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323955
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
repo sync only handles a git tag properly when it is in the "revision"
field. However, "revision locked manifests" (`repo manifest
--revision-as-HEAD`) specifies the tag in the "upstream" field. The
issue is that this tag is not fetched. Only the commit that the tag
points to is fetched. This cases issues as
self._CheckForImmutableRevision() runs and comes to the conclusion that
the tag was changed while in fact, it was just not fetched. This causes
a full sync.
File docs/manifest-format.md, section Element-project:
> Attribute upstream: Name of the Git ref in which a sha1 can be found.
Used when syncing a revision locked manifest in -c mode to avoid having
to sync the entire ref space. Project elements not setting their own
upstream will inherit this value.
Change-Id: I0507d3a5f30aee8920a9f820bafedb48dd5db554
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323620
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Robin Schneider <ypid@riseup.net>
objdir is the .repo/project-objects/ dir based on the remote path.
gitdir is the .repo/projects/ dir based on the local source checkout
path. When we setup the gitdir, we symlink "hooks" to the one in the
objdir. But when we go to initialize the hooks, we do it via gitdir.
There is a 1-to-many mapping from project-objects to projects, so
initializing via gitdir can be repetitive. Collapse the hook init
logic to the objdir init path.
Change-Id: I828fca60ce6e125d6706c709cdb2797faa40aa50
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323815
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This should make it easy to discover for people poking around .repo/.
Change-Id: Ie5051551f25127c0592df5e36efba7bb2263e5d4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323701
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This is used by the `git review` tool that some people use.
Change-Id: I8dac4e1dad155109a05181deaec61e1a74857b1f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323698
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
which will cause error log like below:
NameError: name 'ValueErrorl' is not defined
Change-Id: I388886b7cf6d700e224c3847b7ba4ba4fe9c041d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323015
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: 彭杨益 <pyy101727@gmail.com>
If a remote deletes a ref, and it points to an object that doesn't
exist locally, we can get into a bad state, and the only way for the
user to recover is to run `repo sync --prune` (and to know that is
the option they need). The error message is not helpful:
fatal: bad object refs/remotes/cros/firmware-zork-13421.B-master
error: https://chromium.googlesource.com/chromiumos/platform/ec did not send all necessary objects
This situation can also come up when the remote renames refs in a
UNIX FS incompatible way. For example, replacing refs/heads/foo
with refs/heads/foo/bar.
Also add a --no-prune option for users to disable the behavior.
Bug: https://issuetracker.google.com/203366450
Change-Id: Icf45d838a10938feb091d29800f7e49240830ec3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322956
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This has been broken since it was added where --tags was actually
the same as --no-tags. Oddly, it was copied from init where the
logic is correct.
Bug: https://crbug.com/gerrit/12401
Change-Id: I15b89da1a655176a11bebc22573b25c728055328
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322955
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
urllib.requests.urlopen also supports file, so call it unless the
scheme is 'gs'. This adds http, https, and ftp support.
Change-Id: I3f215c3ebd8e6dee29ba14c7e79ed99d37287109
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322095
Reviewed-by: Michael Kelly <mkelly@arista.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Matt Story <mstory@arista.com>
Previously gsutil stderr was getting piped into stdout, which
yields bad results if there are non-fatal warnings in stderr.
Additionally, we should fail outright if gsutil fails (by adding
`check = True`) rather than fail later on when we try to sync to
a manifest that is in fact just a stderr dump.
BUG=none
TEST=manual runs with bad gs urls
Change-Id: Id71791d0c3f180bd0601ef2c783a8e8e4afa8f59
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321935
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
sync should not attempt to sync the manifest project if it was
created from a standalone manifest. The current work around is to
run sync with --nmu.
BUG=none
TEST=manual runs
Change-Id: I2e121af0badf9642143e77c7af89d1c2d993b0f3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321195
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Previously `git gc` was being run on every gitdir even when they shared
the same objects. Instead only call it once and use pack-refs for the
gitdirs that were not gc'ed.
Bug: https://crbug.com/gerrit/15113
Test: repo sync -j # and check that git pack-refs is called
Change-Id: Icff37ab3ec78cfb44391d8cc7f2d875991532320
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320275
Tested-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
`repo init` doesn't do anything on re-init when the checkout has
been initialized using --standalone manifest. Rather than let the
tool run through its existing flows (which happen to noop), check
the args and explicitly quit if a bare `repo init` is run on a
standalone checkout.
BUG=none
TEST=manual tests
Change-Id: Ie4346ef6df1282ec3e3f8045a08138c93653fece
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320735
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
We were accidentally always setting manifest.standlone in config,
which was messing up behavior for standard use cases.
BUG=gerrit:15160
TEST=manual runs
Change-Id: Ic80f084ae97de5721aced3bb52d3ea9115f8d833
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320715
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Rather than display "3.0" all the time and confuse people, extract
the version from the launcher we're signing and display that.
Also reformat the text to follow our current practice: upload the
versioned launcher by itself first, and then later copy that over
the default.
And while we're here, add tips for rollbacks.
Change-Id: I1654425c88e5c67d78879f2f33ad685c59be14dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319637
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
gc.autoDetach is enabled by default which makes 'git gc --auto' return
immediately and run in background. This can lead to a pile up of
operations all using large amounts of memory at the same time. To avoid
this set gc.autoDetach to false so that the garbage collect task waits
for instances to finish before spawning more.
Bug: https://crbug.com/gerrit/15113
Test: repo sync -j # and check the number of 'git gc' processes
Change-Id: Ic0815156ba3db03972968f33f6f9f51e4928f23b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319835
Tested-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
The v2.14.[345] releases were cut on a branch based on v2.14.2. We
had some regression fixes we wanted in v2.14, but too many risky
changes landed in main since to cut another v2.14.x directly, and
we didn't want to destabilize even more by pushing a v2.15 right
away. So we branched to keep things healthy.
But people with old checkouts trying to upgrade from those versions
run into an old repo bug where it only selfupdates with fast-forwards,
and repo can't fast-forward from those divergent histories. So let's
do a merge commit to stitch the history back together.
There's no actual changes in here.
Change-Id: I05a96048e3846321e57c5f5224fb8dcf3c191d35
For older versions of repo, this would make it easier for it to perform
a self update by making it a fast-forward from the following tags:
v1.13.9.2, v1.13.9.3, v1.13.9.4, v1.13.10, v1.13.11
Change-Id: Ia75776312eaf802a150db8bd7c0a6dce57914580
removeprefix is available i python 3.9. Mac and Linux are running in
a version below 3.9. Thus tests are failing with the following error:
"AttributeError: 'str' object has no attribute 'removeprefix' "
Replaced the removeprefix with custom function which we will delete
once Linux and macOS versions are updated.
Tested:
$ ./run_tests
Bug: [google internal] b/201453085
Change-Id: I9b4d564ff1176e1b4471805ef05472c1914cd9f9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319375
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Now that _do throws an exception when `git` fails, update the logic
that tries to read config files but the file doesn't exist.
Bug: b/192664812
Change-Id: I6417ecd70891b8f2d5f2bdb819f91df69ac4b70c
Test: `repo upload` no longer crashes when .repo/config doesn't exist
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319295
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
When a hard revision ID is provided in a regular project tag then the
revisionId is updated as well if it is a commit hash. The difference
is that if the revisionExpr is a commit, git-repo needs to update
refs/remotes/m/master with update-ref not symbolic-ref, as the latter
must refer to another ref, not to a specific commit.
Change-Id: I215a62dabb30225e480ad2c731416d775fc0c750
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310963
Tested-by: Michael Kelly <mkelly@arista.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
This allows us to move the repository to a new location in the source
tree without having to remove-project + add a new project tag.
Change-Id: I4dba6151842e57f6f2b8fe60cda260ecea68b7b4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/310962
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Michael Kelly <mkelly@arista.com>
The current code would check for enrollment status when the user did not
explicitly specify --[no-]use-superproject and do not have a remembered
value in their repo client. However, because superproject only makes
sense for manifests that have one specified, we should skip the
enrollment logic in that case.
Address this by checking manifest.superproject prior to proceeding. This
would avoid showing the greeting message of superproject enrollment
which can be confusing for developers.
Tested:
For manifest without superproject:
- repo sync --use-superproject will still show message for
superproject;
- repo sync will not show message regardless of enrollment state
For manifest with superproject:
- repo sync will show message and perform enrollment if not
previously enrolled
Bug: https://crbug.com/gerrit/15039
Change-Id: Ic2be9f9d037f0e7cf3446da474a5a0d0e4bd88da
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319255
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
All the values of syncstate are strings, check the first byte and last
byte to see if it is an array. For syncstate data, there were no false
positives.
Tested:
$ repo_dev sync
Verified event logged for argv is "data-json".
$./run_tests
Bug: [google internal] b/201102002
Change-Id: Id56adb532b80267f08d09147ac663cdd5987ce87
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319075
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
Some of the file removal calls are subject to race conditions (if
something else deletes the file), so extend our remove API to have
an option to ignore ENOENT errors. Then update a bunch of random
call sites to use this new functionality.
Change-Id: I31a9090e135452033135337a202a4fc2dbf8b63c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319195
Reviewed-by: Sean McAllister <smcallis@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Added --standalone_manifest to repo tool. If set, the
manifest is downloaded directly from the appropriate source
(currently, we only support GS) and used instead of creating
a manifest git checkout. The manifests.git repo is still created to
keep track of various config but is marked as being for a standalone
manifest so that the repo tool doesn't try to run networked git
commands in it.
BUG=b:192664812
TEST=existing tests (no coverage), manual runs
Change-Id: I84378cbc7f8e515eabeccdde9665efc8cd2a9d21
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312942
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Fixed: "we need to make a special case for logging the argv; it
should probably be a "data-json" event so that we log this directly as
an array rather than an encoded string.
Tested:
$ repo_dev sync
Verified event logged for argv is "data-json".
$./run_tests
Bug: [google internal] b/201102002
Change-Id: I18ccec79c73c8dc931cb8afc472b2361db8aea4c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319055
Reviewed-by: Josh Steadmon <steadmon@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
Tested:
$ ./run_tests
Verified that a manifest that specified superproject revision would use
the specified revision, and superproject will use the default revision.
Note that this is a slight behavior change from earlier repo versions,
which would always use the branch name of the manifest itself. However,
the new behavior would be more consisitent with regular "project"
element and would allow superproject be used even if it is not enabled
for the particular manifest branch, so we have decided to make the
change as it would provide more flexibility and better matches what
other elements would do.
Bug: [google internal] b/187868160
Change-Id: I35255ee347aff6e65179f7879d52931f168b477e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317643
Tested-by: Xin Li <delphij@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Stop tracking Ubuntu Trusty & Xenial and Debian Jessie & Stretch
as they only had Python 3.5 available which we've dropped.
Backfill OpenSSH versions since we've started testing for it.
Change-Id: I03183ed97f6e43dce8a00e36cce2956544a26afc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/318835
Reviewed-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
