Commit Graph

1855 Commits

Author SHA1 Message Date
LaMont Jones
d56e2eb421 manifest_xml: use Superproject to hold XML content
Always create Superproject when there is a <superproject> tag, and have
it hold the XML content, similar to how other manifest elements are
handled.

This also adds SetQuiet and SetPrintMessages to Superproject
consistent with manifest.SetUseLocalManifests.

Change-Id: I522bf3da542006575799f0640c67f7052704f266
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334641
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-04-12 15:46:23 +00:00
Daniel Andersson
d52ca421d5 sync: respect sync-c manifest option
The documentation states that a `sync-c` attribute in the manifest file
can set a default for whether only the current branch should be fetched
or all branches. This seems to have been broken for some time.

Commit 7356114 introduced the `--no-current-branch` CLI option and
relied on getting `None` via `optparse` if neither `--current-branch`
nor `--no-current-branch` was set to distinguish it from a boolean
value. If `None` was received, it would read the value from the manifest
option `sync-c`. The parsing went through the utility function
`_GetCurrentBranchOnly` which returned `True` if `--current-branch` had
been given on the command-line, or fell back on the "superproject"
setting, which would either return `True` or `None`. This would
incorrectly make `repo` fall back to the manifest setting even if the
user had given `--no-current-branch` if no superproject was requested --
the manifest became "too powerful":

Command-line         Using superproject  → `current_branch_only`
------------         ------------------  -----------------------
                     No                  From manifest
                     Yes                 True
--current-branch     No                  True
--current-branch     Yes                 True
--no-current-branch  No                  From manifest ← wrong
--no-current-branch  Yes                 True

In commit 0cb6e92 the superproject configuration value reading changed
from something that could return `None` to something that always
returned a boolean. If it returned `False`, this would then incorrectly
make `repo` ignore the manifest option even if neither
`--current-branch` nor `--no-current-branch` had been given. The
manifest default became useless:

Command-line         Using superproject  → `current_branch_only`
------------         ------------------  -----------------------
                     No                  False ← wrong
                     Yes                 True
--current-branch     No                  True
--current-branch     Yes                 True
--no-current-branch  No                  False
--no-current-branch  Yes                 True

By swapping the order in which the command-line option target and the
superproject setting is evaluated, things should work as documented:

Command-line         Using superproject  → `current_branch_only`
------------         ------------------  -----------------------
                     No                  From manifest
                     Yes                 True
--current-branch     No                  True
--current-branch     Yes                 True
--no-current-branch  No                  False
--no-current-branch  Yes                 True

Change-Id: I933c232d2fbecc6b9bdc364ebac181798bce9175
Tested-by: Daniel Andersson <daniel.r.andersson@volvocars.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334270
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-08 21:06:37 +00:00
LaMont Jones
a2ff20dd20 manifest_xml: Add Load and Unload methods
- do not call the internal method from subcmds/sync.py.
- use the correct default groups for submanifests.
- only sync the superproject when we are told to.

Change-Id: I81e4025058f1ee564732b9e17aecc522f6b5f626
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334639
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-04-08 19:52:04 +00:00
LaMont Jones
55ee304304 Fix sub manifest handling
Also fixes some typos

Change-Id: Id2ba5834ba3a74ed3f29c36d2c0030737dc63e35
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334579
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-06 21:04:46 +00:00
LaMont Jones
409407a731 init: add multi-manifest support
This moves more of the manifest project handling into ManifestProject.

Change-Id: Iecdafbec18cccdfd8e625753c3bd1bcddf2b227f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334520
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-06 17:02:40 +00:00
LaMont Jones
d82be3e672 Move manifest config logic into ManifestProject
Use ManifestProject properties for config values.

Change-Id: Ib4ad90b0d9a089916e35615b8058942e6d01dc04
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334519
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-06 16:59:45 +00:00
LaMont Jones
9b03f15e8e project: add ManifestProject.Sync()
Move the logic to sync a ManifestProject out of subcmds/init.py

Change-Id: Ia9d00f3da1dc3c5dada84c4d19cf9802c2346cb0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334140
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-01 15:48:04 +00:00
LaMont Jones
9b72cf2ba5 project: Isolate ManifestProject from RepoProject
Create RepoProject and ManifestProject, inheriting from MetaProject,
  with methods separated for isolation and clarity.

Change-Id: Ic1d6efc65c99470290fea612e2abaf8670d199f4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334139
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-03-31 21:02:52 +00:00
LaMont Jones
5d3291d818 manifest_file must be an absolute path
Correctly pass the full path of the manifest file for the submanifest.
The manifest-name in the <submanifest/> element was being passed in
as given, which caused it to not be found since the current directory
never set. (b/226333721: fails when manifest-name is given.)

Also verify that the manifest_file passed to XmlManifest() is an
absolute path.

Bug: https://b.corp.google.com/issues/226333721
Change-Id: I23461078233e34562bc2eafeb732cfe8bd38ddc1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/333861
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-03-23 21:18:41 +00:00
Josh Steadmon
244c9a71a6 trace: allow writing traces to a socket
Git can write trace2 events to a Unix domain socket [1]. This can be
specified via Git's `trace2.eventTarget` config option, which we read to
determine where to log our own trace2 events. Currently, if the Git
config specifies a socket as the trace2 target, we fail to log any
traces.

Fix this by adding support for writing to a Unix domain socket,
following the same specification that Git supports.

[1]: https://git-scm.com/docs/api-trace2#_enabling_a_target

Change-Id: I928bc22ba04fba603a9132eb055141845fa48ab2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/332339
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Josh Steadmon <steadmon@google.com>
2022-03-16 17:33:07 +00:00
LaMont Jones
b308db1e2a manifest_xml: group for submanifest projects
Add all projects in a submanifest to the group
submanifest::<path_prefix> for ease in filtering.

Change-Id: Ia6f01f9445f4f8d20fda3402f4d5821c43ceaf7f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/331319
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-02-28 20:08:58 +00:00
LaMont Jones
cc879a97c3 Add multi-manifest support with <submanifest> element
To be addressed in another change:
 - a partial `repo sync` (with a list of projects/paths to sync)
   requires `--this-tree-only`.

Change-Id: I6c7400bf001540e9d7694fa70934f8f204cb5f57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322657
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-02-17 21:57:55 +00:00
LaMont Jones
87cce68b28 Move local-manifest check to manifest_xml.py
This removes the need for git_superproject to include manifest_xml, and
puts the logic for local_manifest detection in one place.

Change-Id: I4d33ded0542ceea4606a1ea24304f678de20c59e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/330499
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
2022-02-15 22:15:42 +00:00
LaMont Jones
adaa1d8734 project.py: pass --recurse-submodules={value}
If submodules is False, explicitly pass '=no'.  Uninitialized submodules
may cause the default option to fail.

Change-Id: Ia00bcba5b69c4b65195f4c469c686a3ef9a4a3ad
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/330159
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-02-10 23:57:45 +00:00
Mike Frysinger
8e91248655 project: mark gc.log as safe to discard when migrating .git/
This is just a log file that, while useful for humans when gc aborts,
doesn't contain any data, so it's safe to throw away.

Bug: https://crbug.com/gerrit/15619
Change-Id: Ia95e0e281f52260668f7a80b5d5f990e32a8597a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328999
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-01-26 16:46:03 +00:00
XD Trol
630876f9e4 init: add an option --enable-git-lfs-filter
It was reported that git-lfs did not work with git-repo. Specifically,
`git read-tree -u` run by `repo sync` would fail git-lfs's smudge
filter. See https://github.com/github/git-lfs/issues/1422.

In fact, by the time `git read-tree -u` is run, the repository is not
bare. It is just that, the working directory is not the same as the
.git directory. git-lfs's filter should work. No one seems to have
delved into that issue.

Today, with newer versions of git-repo and git-lfs, that issue will
not reproduce. Tested with
- git 2.33, git-lfs 2.13 on macOS
- git 2.17, git-lfs 2.3 on ubuntu

So, it seems fine to add an option --enable-git-lfs-filter, default to
false, and stat that it may not work with older versions of git and
git-lfs in the help doc.

Bug: https://crbug.com/gerrit/14516
Change-Id: I8d21854eeeea541e072f63d6b10ad1253b1a9826
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328359
Tested-by: XD Trol <milestonejxd@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-01-26 01:47:20 +00:00
Mike Frysinger
4aa8584ec6 init: make bad --repo-rev settings more clear
If the user passes a bad --repo-rev setting in a new checkout, add a
tip to the error message that their option is probably bad instead of
just saying "unable to resolve".

If the user has already initialized a checkout, we'd display a raw
traceback which would confuse them.  Swallow that and also include
the --repo-rev tip.

Bug: https://crbug.com/gerrit/15610
Change-Id: I5d72513c7b37bf9bb5d19862fcdfaf0d1f44e886
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328820
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-25 18:54:42 +00:00
Peter Kjellerstedt
b550501254 project: Ignore failure to remove the sample hooks
Removing the sample hooks is just clean up, so if repo cannot remove a
sample hook that should not cause it to fail.

Change-Id: I716b977da091c22b8f53e134f4fbc114116f9a65
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328635
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-01-22 06:13:10 +00:00
LaMont Jones
a535ae4418 branches: Fix "not in" handling
If the branch is current, or present in less than half of the projects,
list which projects it is *in*.

Otherwise, correctly detect which projects (by relpath) it is not in.

Previously, the "not in" path would incorrectly list all projects.

Change-Id: Ia153856f577035a51f538b7bf5d3135b70c69d52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328199
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
2022-01-21 17:20:21 +00:00
Mike Frysinger
67d6cdf2bc project: store objects in project-objects directly
In order to stop sharing objects/ directly between shared projects,
we have to fetch the remote objects into project-objects/ manually.
So instead of running git operations in the individual project dirs
and relying on .git/objects being symlinked to project-objects/,
tell git to store any objects it fetches in project-objects/.

We do this by leveraging the GIT_OBJECT_DIRECTORY override.  This
has been in git forever, or at least since v1.7.2 which is what we
already hard require.  This tells git to save new objects to the
specified path no matter where it's being run otherwise.

We still otherwise run git in the project-specific dir so that it
can find the right set of refs that it wants to compare against,
including local refs.  For that reason, we also have to leverage
GIT_ALTERNATE_OBJECT_DIRECTORIES to tell git where to find objects
that are not in the upstream remote.  This way git doesn't blow up
when it can't find objects only associated with local commits.

As it stands right now, the practical result is the same: since we
symlink the project objects/ dir to the project-objects/ tree, the
default objects dir, the one we set $GIT_OBJECT_DIRECTORY to, and
the one we set $GIT_ALTERNATE_OBJECT_DIRECTORIES to are actually
all the same.  So this commit by itself should be safe.  But in a
follow up commit, we can replace the symlink with a separate dir
and git will keep working.

Bug: https://crbug.com/gerrit/15553
Change-Id: Ie4e654aec3e1ee307eee925a54908a2db6a5869f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328100
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-19 17:24:51 +00:00
Mike Frysinger
152032cca2 project: move --reference handling to project-objects
When using --reference, the path is written to objects/info/alternates.
The path is accessed inconsistently -- sometimes through projects/ (via
self.gitdir) and sometimes through project-objects/ (via self.objdir).
This works because projects/.../objects is a symlink to the objects dir
under project-objects/.  Change all accesses to go through self.objdir.
This will allow us to stop symlinking projects/.../objects without the
reference dir logic breaking.  The projects/ path is going to use its
alternates file for its own needs.

Bug: https://crbug.com/gerrit/15553
Change-Id: I6b452ad1aaffec74ecb7ac1bb9baa3a3a52e076c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328099
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jack Neus <jackneus@google.com>
2022-01-13 18:27:28 +00:00
Sebastian Wagner
a3ac816278 test_project: use os.readlink instead of Path.readlink
Path.readlink is only available on Python 3.9, breaking compatibility
with all python versions below. os.readlink is already used in other
places of this file, so use it here as well.

Change-Id: I5acf8f5334a3e7c8de9cea1939d7e2b9af5f30ae
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327844
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Sebastian Wagner <sebix@sebix.at>
2022-01-11 20:16:42 +00:00
Mike Frysinger
98bb76577d project: prune sample hooks
These hooks are never used and often get stale, so just trim them.
Users rarely look in these dirs to begin with.

Change-Id: Ic785aa55fb7ec84a61376df101127d0018882030
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327538
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-10 17:41:45 +00:00
Mike Frysinger
d33dce0b77 project: drop support for symlinking internal .git files
Since we don't do this anymore, and there prob won't be a need to
bring it back, drop support for it.

Bug: https://crbug.com/gerrit/15460
Change-Id: I7d86706f108c797a5c7962cb1578693d49430367
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327537
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-10 17:41:40 +00:00
Mike Frysinger
89ed8acdbe project: abort a bit earlier before migrating .git/
Verify all the .git/ paths will be handled by the migration logic before
starting the migration.  This way we still abort & log an error, but the
user gets to see it before we put the tree into a state that they have to
manually recover.  Also add a few more known-safe-to-clobber paths.

Bug: https://crbug.com/gerrit/15273
Change-Id: If49d69b341bc960ddcafa30da333fb5ec7145b51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327557
Reviewed-by: Colin Cross <ccross@android.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-07 20:17:14 +00:00
Raman Tenneti
71e48b7672 Revert "sync: dropped "NOTICE: --use-superproject is in beta ..." message."
This reverts commit d53cb9549a. As long as
repo's reference docs treat this feature as a work in progress and don't
cover it well enough to allow all repo maintainers to easily support it,
it is inconsistent to report to users that it is no longer in beta.
Thanks for vapier@google.com for noticing.

https://crbug.com/gerrit/15527 tracks the required documentation changes
before we'd be ready to roll forward again.

Change-Id: Ic9bd951cfb3c1abf6e1bfa30dfe4afa1c9b7bec6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327337
Reviewed-by: Jonathan Nieder <jrn@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Jonathan Nieder <jrn@google.com>
2022-01-06 20:01:03 +00:00
Mike Frysinger
13576a8caf project: stop symlinking info dir under .git/
Unsharing this directory shouldn't be a problem.  The current repo code
treated it as a file, and while that's actually incorrect, files & dirs
are basically treated the same, so it's practically the same.

Let's enumerate each subpath since there aren't that many.

info/refs:
Only used when the project is exported over git dumb transports (i.e.
a http:// server).  Repo never does this, and it's extremely unlikely
any user has ever done this.  Plus, this proposal talks about unsharing
project refs, so this file should get unshared too.

info/grafts:
A user-configurable file that repo never touches.  Might be useful to
share across projects, but probably rarely (if ever) used by developers,
and forcing them to configure it for each project isn't that big of a
deal.

info/exclude:
info/attributes:
User-configurable files that repo never touches.  Doesn't seem like
most users ever touch these, and if they do, having them do it for
each shared project isn't a big deal.

info/sparse-checkout:
Repo doesn't use sparse checkouts, and it's extremely unlikely to even
work if a user tried doing something themselves.

Bug: https://crbug.com/gerrit/15460
Change-Id: I53e44d73a6d7a92da615b46600d8ea51cb46e3ac
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327519
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-06 08:31:45 +00:00
Mike Frysinger
2345906d04 project: stop symlinking description file under .git/
Nothing uses this path.  It’s only for exporting git dirs e.g. for
online gitweb use which probably no one does.  It is not the same
description file as exists on servers we cloned from.  Leaving it
as the default plain text file will simplify code.

We don't undo any existing symlinks if they exist since repo does
not care about them, and their existence doesn't hurt.

Bug: https://crbug.com/gerrit/15460
Change-Id: Ic34fe7c3cfb8f6da844de5be30158f59382b1cc8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327518
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-06 08:29:06 +00:00
Mike Frysinger
41289c62b4 project: stop symlinking svn under .git/
This path only matters to users of `git svn` who manually run it in
local projects after they get a full repo client checkout.  With svn
usage falling in general, and with the fact that the source checkout
now symlinks its .git/ state to the internal projects/ path, we don't
need to manage this anymore.

It means the path won't be shared among multiple local projects that
have the same remote, but so it goes.  It was an optimization only,
not functionality required for correctness.  We want to simplify the
internals to stop messing with git state, and this particular path
doesn't seem worth the effort to maintain.

We don't undo any existing svn symlinks if they exist since repo does
not care about them, and their existence doesn't hurt anything.

Bug: https://crbug.com/gerrit/15460
Change-Id: Ie8496b275bcc589771aa9f4ee874ed2ee6d5241d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327517
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-06 08:28:37 +00:00
Mike Frysinger
c72bd8486a project: clean up now unused code
Now that we symlink worktree .git/ paths to .repo/projects/, we never
set share_refs=True anywhere, which means all of this logic is dead
code.  Throw it all away.  Do it as a separate commit to make the
parent commit easier to review.

Bug: https://crbug.com/gerrit/15273
Change-Id: If496d39029d3d3bd523ba24c603ce47a63ad9b51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/326817
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jack Neus <jackneus@google.com>
2022-01-06 04:08:05 +00:00
Raman Tenneti
d53cb9549a sync: dropped "NOTICE: --use-superproject is in beta ..." message.
Tested the code with the following commands.

$ ./run_tests -v

Bug: [google internal] b/209511230
Change-Id: Ia3c6de47709f5276e324a5bb608383aba3b2c562
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/327197
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-12-29 19:07:08 +00:00
Raman Tenneti
cf0ba48649 sync: With --mirror option, don't display no-use-superproject... message.
+ Display 'Defaulting to no-use-superproject because there is no working tree.'
  message if --use-superproject option is used and we are not using
  superproject because manifest is either a mirror or is an archive.

Tested the code with the following commands.

$ ./run_tests -v

Tested the sync code by using repo_dev alias and pointing to this CL.

$ repo init -u https://android.googlesource.com/mirror/manifest --mirror

$ repo_dev sync
Receiving objects: 100% (3/3), done.eiving objects:  33% (1/3)

$ repo_dev sync --use-superproject
Defaulting to no-use-superproject because there is no working tree.
Fetching:  0% (0/2158) warming up

Bug: https://crbug.com/gerrit/15368
Change-Id: I16b87ee9623315dbc3100b612b1decdaab7ac1dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/325797
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-12-07 16:46:41 +00:00
Mike Frysinger
2a089cfee4 project: migrate worktree .git/ dirs to symlinks
Historically we created a .git/ subdir in each source checkout and
symlinked individual files to the .repo/projects/ paths.  This layer
of indirection isn't actually needed: the .repo/projects/ paths are
guaranteed to only ever have a 1-to-1 mapping with the actual git
checkout.  So we don't need to worry about having files in .git/ be
isolated.

To that end, change how we manage the actual project checkouts from
a dir full of symlinks (and a few files) to a symlink to the internal
.repo/projects/ dir.  This makes the code simpler & faster.

The directory structure we have today is:
.repo/
  project-objects/chromiumos/third_party/kernel.git/
    <paths omitted as not relevant to this change>
  projects/src/third_party/kernel/
    v3.8.git/
      config
      description   -> …/project-objects/…/config
      FETCH_HEAD
      HEAD
      hooks/        -> …/project-objects/…/hooks/
      info/         -> …/project-objects/…/info/
      logs/
      objects/      -> …/project-objects/…/objects/
      packed-refs
      refs/
      rr-cache/     -> …/project-objects/…/rr-cache/
src/third_party/kernel/
  v3.8/
    .git/
      config        -> …/projects/…/v3.8.git/config
      description   -> …/project-objects/…/v3.8.git/description
      HEAD
      hooks/        -> …/project-objects/…/v3.8.git/hooks/
      index
      info/         -> …/project-objects/…/v3.8.git/info/
      logs/         -> …/projects/…/v3.8.git/logs/
      objects/      -> …/project-objects/…/v3.8.git/objects/
      packed-refs   -> …/projects/…/v3.8.git/packed-refs
      refs/         -> …/projects/…/v3.8.git/refs/
      rr-cache/     -> …/project-objects/…/v3.8.git/rr-cache/

The directory structure we have after this commit:
.repo/
  <nothing changes>
src/third_party/kernel/
  v3.8/
    .git            -> …/projects/…/v3.8.git

Bug: https://crbug.com/gerrit/15273
Change-Id: I9dd8def23fbfb2f4cb209a93f8b1b2b24002a444
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323695
Reviewed-by: Mike Nichols <mikenichols@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-12-01 15:27:16 +00:00
Raman Tenneti
4a478edb44 init, sync: fixed flake8 warnings.
Tested:
+ run_tests
+ flake8 subcmds/init.py
+ flake8 subcmds/sync.py

Change-Id: Ie337481d8a210bfc49b0745f75c05a308a0e74d3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/324155
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-18 16:22:40 +00:00
Raman Tenneti
6bd89aa657 superproject: Inherit --no-use-superproject with --mirror option.
init.py
+ Similar to opt.archive, gave an error if --mirror option is
  used with --use-superproject.

sync.py
+ Defaulted to --no-use-superproject if manifest is a mirror or
  archive (similar to error at line# 1067).

Tested:
+ run_tests
+ flake8 (will fix known errors in another CL).

$ repo_dev init -u sso://googleplex-android.git.corp.google.com/platform/manifest --use-superproject --mirror
Usage: repo init [options] [manifest url]

main.py: error: --mirror and --use-superproject cannot be used together.

+ repo init and repo sync with --mirror and without --mirror
  options.
  $ repo_dev init -u https://android.googlesource.com/platform/manifest
  $ repo_dev sync
    ...superproject.git: Initial setup for superproject completed.

+ With --mirror option, verfied there are no exceptions in git_superproject.py

Bug: [google internal] b/206537893
Change-Id: I059f20e76f0ab36f0587f29779bb53ede4663bd4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323955
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-18 01:27:41 +00:00
Robin Schneider
9c1fc5bc5d sync: Handle tag ref in "upstream" field
repo sync only handles a git tag properly when it is in the "revision"
field. However, "revision locked manifests" (`repo manifest
--revision-as-HEAD`) specifies the tag in the "upstream" field. The
issue is that this tag is not fetched. Only the commit that the tag
points to is fetched. This cases issues as
self._CheckForImmutableRevision() runs and comes to the conclusion that
the tag was changed while in fact, it was just not fetched. This causes
a full sync.

File docs/manifest-format.md, section Element-project:
> Attribute upstream: Name of the Git ref in which a sha1 can be found.
Used when syncing a revision locked manifest in -c mode to avoid having
to sync the entire ref space. Project elements not setting their own
upstream will inherit this value.

Change-Id: I0507d3a5f30aee8920a9f820bafedb48dd5db554
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323620
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Robin Schneider <ypid@riseup.net>
2021-11-16 20:43:57 +00:00
Mike Frysinger
333c0a499b project: init hooks in objdir only
objdir is the .repo/project-objects/ dir based on the remote path.
gitdir is the .repo/projects/ dir based on the local source checkout
path.  When we setup the gitdir, we symlink "hooks" to the one in the
objdir.  But when we go to initialize the hooks, we do it via gitdir.
There is a 1-to-many mapping from project-objects to projects, so
initializing via gitdir can be repetitive.  Collapse the hook init
logic to the objdir init path.

Change-Id: I828fca60ce6e125d6706c709cdb2797faa40aa50
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323815
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 19:50:18 +00:00
Mike Frysinger
fdeb20f43f sync: link the internal-fs-layout doc into checkouts
This should make it easy to discover for people poking around .repo/.

Change-Id: Ie5051551f25127c0592df5e36efba7bb2263e5d4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323701
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 01:39:53 +00:00
Mike Frysinger
bf40957b38 git-review: add config file
This is used by the `git review` tool that some people use.

Change-Id: I8dac4e1dad155109a05181deaec61e1a74857b1f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323698
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 01:39:36 +00:00
Mike Frysinger
f9e81c922d SUBMITTING_PATCHES: link to commit message style docs
Change-Id: I2090ebc43fc1c816b941a53dd89dbedf7bc61289
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323696
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jack Neus <jackneus@google.com>
2021-11-15 01:39:16 +00:00
Mike Frysinger
e6601067ed man: refresh pages
Change-Id: I3f2c3ad77c16a76276bba2954887ab9e7605661c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323516
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-12 17:30:45 +00:00
Mike Frysinger
3001d6a426 help: fix grammar in help text
Bug: https://crbug.com/gerrit/14838
Change-Id: Ic5000921ba9a1baa086153630ebbb429e3d17642
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323515
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-12 17:30:32 +00:00
Sl0v3C
00c5ea3787 Fix typo for ValueError
which will cause error log like below:
NameError: name 'ValueErrorl' is not defined

Change-Id: I388886b7cf6d700e224c3847b7ba4ba4fe9c041d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323015
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: 彭杨益 <pyy101727@gmail.com>
2021-11-07 02:32:08 +00:00
Mike Frysinger
0531a623e1 sync: make --prune the default
If a remote deletes a ref, and it points to an object that doesn't
exist locally, we can get into a bad state, and the only way for the
user to recover is to run `repo sync --prune` (and to know that is
the option they need).  The error message is not helpful:

fatal: bad object refs/remotes/cros/firmware-zork-13421.B-master
error: https://chromium.googlesource.com/chromiumos/platform/ec did not send all necessary objects

This situation can also come up when the remote renames refs in a
UNIX FS incompatible way.  For example, replacing refs/heads/foo
with refs/heads/foo/bar.

Also add a --no-prune option for users to disable the behavior.

Bug: https://issuetracker.google.com/203366450
Change-Id: Icf45d838a10938feb091d29800f7e49240830ec3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322956
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-05 20:13:30 +00:00
Mike Frysinger
2273f46cb3 sync: fix --tags option
This has been broken since it was added where --tags was actually
the same as --no-tags.  Oddly, it was copied from init where the
logic is correct.

Bug: https://crbug.com/gerrit/12401
Change-Id: I15b89da1a655176a11bebc22573b25c728055328
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322955
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-05 20:13:02 +00:00
LaMont Jones
7b9b251a5e project: fix format string in error message
BUG=None

Change-Id: I0b195fd919c6db8cb3547e8d6f4c733f2bd4a535
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322735
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
2021-11-05 16:59:14 +00:00
Raman Tenneti
6251729cb4 superproject: added 'implies -c' in the help of --use-superproject option.
sync.py: deleted unused import errno.

Tested:
$ ./run_tests
$ flake8 repo subcmds/sync.py

Bug: https://crbug.com/gerrit/15208
Change-Id: I2bb3098f5602ded3861e000100766041ad93b53d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322555
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-01 22:15:53 +00:00
Matt Story
11b30b91df Support more url schemes for getting standalone manifest
urllib.requests.urlopen also supports file, so call it unless the
scheme is 'gs'.  This adds http, https, and ftp support.

Change-Id: I3f215c3ebd8e6dee29ba14c7e79ed99d37287109
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322095
Reviewed-by: Michael Kelly <mkelly@arista.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Matt Story <mstory@arista.com>
2021-10-27 13:20:35 +00:00
Jack Neus
198838599c fetch: Fix stderr handling for gsutil
Previously gsutil stderr was getting piped into stdout, which
yields bad results if there are non-fatal warnings in stderr.

Additionally, we should fail outright if gsutil fails (by adding
`check = True`) rather than fail later on when we try to sync to
a manifest that is in fact just a stderr dump.

BUG=none
TEST=manual runs with bad gs urls

Change-Id: Id71791d0c3f180bd0601ef2c783a8e8e4afa8f59
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321935
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-26 22:18:28 +00:00
Mike Frysinger
282d0cae89 ssh: handle FileNotFoundError errors
If ssh isn't installed, it throws a distinct error we have to catch.

Bug: https://crbug.com/gerrit/15196
Change-Id: I0660e842c304ce7575f5cb100894d05fd65f9454
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322055
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-10-26 16:18:45 +00:00