Commit Graph

661 Commits

Author SHA1 Message Date
Mike Frysinger
488d54d4ee init: show a notice when reinitializing
Make it clear to users when we're reinitializing an existing checkout
in case they weren't expecting it.

Bug: https://crbug.com/gerrit/12396
Change-Id: I22e89ae041a8e7b147c9d06a82f1302dd5807ae0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343535
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-08-22 21:01:10 +00:00
Mike Frysinger
5a5cfce1b2 stage: add missing flush before project prompt
Bug: https://crbug.com/gerrit/13223
Change-Id: Ib279d86a52e1035e02d6f7d8f053c3a43e721032
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343555
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-08-22 20:48:29 +00:00
Mike Frysinger
e6d4b84060 upload: respect --yes with large upload confirmation
If the user passes in --yes, don't prompt them to confirm large uploads.

Bug: https://crbug.com/gerrit/14085
Change-Id: Ic801b21be80ba181801531acd4af5057ec10c11c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343554
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-08-22 20:48:27 +00:00
Mike Frysinger
8a54a7eac3 cherry-pick: tighten up output
If stdout or stderr are empty, don't print empty lines.
Also trim any trailing lines so we don't show excess ones.

Change-Id: I65fcea191e1f725be03c064cb510578632466013
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343516
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
2022-08-22 19:38:32 +00:00
Mike Frysinger
076d54652e init: change --depth default to 1 for manifest repo
People rarely care about the history of the manifest repo.  Change
the default to 1 to speed up initial setup.  If people really want
the full history, they can pass --manifest-depth=0.

We can't reuse the existing --depth option because that applies to
all projects we clone, not just the manifest repo.

Bug: https://crbug.com/gerrit/16193
Change-Id: Ideab1712e9ffc743b9bae97903e074d7d429b134
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343435
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
2022-08-18 19:12:21 +00:00
Mike Frysinger
7fa8eedd8f upload: add --push-options tips & doc link
Change-Id: Iee38a80974c53231d1e9f04f7f85b2d0bac96dbb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/342354
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-08-18 16:40:40 +00:00
Mike Frysinger
dede564c3d project: simplify GetRemote a bit
We almost always use self.remote.name when calling self.GetRemote,
so make that the default to simplify the code a bit.

Change-Id: Ifdf6e1370d6b8963b44e6d384b0fac8fa5c4f2ba
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/343184
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
2022-08-17 16:54:21 +00:00
William Escande
ac76fd3e3a upload: Add ready flag to remove wip
The `--wip` allow to bulk push changed as work-in-progress. This CL
intend to allow the opposite opperation by removing the wip mark on the
CL and set it to be ready to review

Change-Id: If0743c5b14829f77be2def5a8547060d06a5648c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/342214
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: William Escande <wescande@google.com>
2022-08-03 20:17:06 +00:00
Mike Frysinger
5951e3043f sync: handle smartsync HTML responses better
If the server responds with an HTML page, we should show that to the
user instead of crashing with XML errors.

Bug: https://crbug.com/gerrit/15936
Change-Id: I52e6b781c3bb6a6c9f6ecbe2e0907044876cdc8d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337519
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-07-28 17:44:21 +00:00
LaMont Jones
48ea25c6a7 sync: start clearing git settings that disabled pruning
For projects that no longer share their per-project objects directly, we
no longer have to disable the git settings that disable pruning.  See
commit "project: stop directly sharing objects/ between shared projects"
for more details.

Bug: https://crbug.com/gerrit/15553
Change-Id: Ica0e83c3002716424c2bc9111b3b3d3a76c30973
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337535
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-07-25 17:13:20 +00:00
Mike Frysinger
355f4398d8 sync: rework --jobs to provide better defaults
For --jobs-network, the logic is now:
* If the user specifies --jobs-network, use that.
* Else, if the user specifies --jobs, use that.
* Else, if the manifest specifies sync-j, use that.
* Else, default to 1.
Then we limit the jobs count based on the softlimit RLIMIT_NOFILE.

For --jobs-checkout, the logic is now:
* If the user specifies --jobs-checkout, use that.
* Else, if the user specifies --jobs, use that.
* Else, if the manifest specifies sync-j, use that.
* Else, default to DEFAULT_LOCAL_JOBS which is based on user's ncpus.
Then we limit the jobs count based on the softlimit RLIMIT_NOFILE.

For garbage collecting, the logic is now:
* If the user specifies --jobs, use that.
* Else, if the manifest specifies sync-j, use that.
* Else, default to the user's ncpus.
Then we limit the jobs count based on the softlimit RLIMIT_NOFILE.

Having to factor in the manifest settings makes this more complicated
which is why we delay processing of defaults until after we've synced
the manifest projects.

Bug: http://b/239712300
Change-Id: Id27cda63c76c156f1d63f6a20cb2c4ceeb3d547c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/341394
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
2022-07-25 15:36:43 +00:00
Mateus Azis
bddc964d93 Fix the printed path of the ".repo" dir after a fresh init.
Apparently, manifest.topdir already contains a trailing slash in some
cases, so a simple string concatenation may or not lead to double
slashes. It is safer to use os.path.join. See
https://screenshot.googleplex.com/6pSY3QewAeCdAqk

Change-Id: I2411452296b7e78fc975787b675273a48d6b3d85
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/341574
Reviewed-by: LaMont Jones <lamontjones@google.com>
Tested-by: Mateus Azis <azis@google.com>
2022-07-25 15:22:14 +00:00
LaMont Jones
8501d4602a status, diff: display correct path for multi-manifest
Display the project path relative to the outermost manifest by default,
and relative to the sub manifest only when --this-manifest-only is
specified.

For project-related diagnostic messages, use the outermost manifest for
messages.

Change-Id: I4537d7dd412a2c182e77d6720e95c1b0ef70eb0e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/340754
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-07-14 16:00:18 +00:00
Martin Geisler
9fb64ae29c upload: add ‘--ignore-untracked-files’ option
This option will suppress the

    Uncommitted changes in ... (did you forget to amend?)

prompt when there are untracked (unknown) files in the working copy.
The prompt is still shown if tracked files are modified.

Change-Id: Ia3fcc82989b7fad09b69214eda31e2d0dfc14600
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/340456
Tested-by: Martin Geisler <mgeisler@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-07-11 17:57:43 +00:00
LaMont Jones
ff6b1dae1e Only sync superproject if it will be used.
If the user says `--no-use-superproject`, then do not bother syncing the
superproject.

Also add/update docstrings and comments throughout.

Change-Id: I9cdad706130501bab9a22d3099a1dae605e9c194
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/338975
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-06-08 16:49:08 +00:00
LaMont Jones
bdcba7dc36 sync: add multi-manifest support
With this change, partial syncs (sync with a project list) are again
supported.

If the updated manifest includes new sub manifests, download them
inheriting options from the parent manifestProject.

Change-Id: Id952f85df2e26d34e38b251973be26434443ff56
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334819
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-05-26 00:03:37 +00:00
Mike Frysinger
3a0a145b0e upload: move label validation to core function
This way we know we don't need to encode the labels.

Change-Id: Ib83ed8f4ed05f00b9d2d06a9dd3f304e4443430e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337518
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
2022-05-21 19:19:44 +00:00
LaMont Jones
501733c2ab manifest: add submanifest.default_groups attribute
When the user does not specify any manifest groups, this allows the
parent manifest to indicate which manifest groups should be used for
syncing the submanifest.

Change-Id: I88806ed35013d13dd2ab3cd245fcd4f9061112c4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335474
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-29 18:42:23 +00:00
LaMont Jones
0de4fc3001 project: Add missing imports
Some imports were missed when moving manifestProject to project.py

Bug: https://bugs.chromium.org/p/gerrit/issues/detail?id=15861
Change-Id: Id8fffeaa3f88f344a13b5ab44e5403c7edd98f31
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335554
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
2022-04-21 18:44:26 +00:00
LaMont Jones
a46047a822 sync: refactor use of self.manifest
We need to iterate over multiple manifests, and generally use the
outer_client.manifest for multi-manifest support.  This refactors the
use of self.manifest into a chosen manifest.

Change-Id: I992f21d610c929675e99555ece9c38df4b635839
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334699
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-14 22:24:04 +00:00
LaMont Jones
5fa912b0d1 Stop passing optparse.Values to git_superproject
Make git_superproject independent of the command line by passing
the specific value instead of requiring the caller to have an
optparse.Values object to pass in.

Flag --use-superproject and --archive as incompatible in subcmds/init.py

Change-Id: Ied7c874b312e151038df903c8af4328f070f387c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335135
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-14 22:23:16 +00:00
LaMont Jones
d8de29c447 forall: fix multi-manifest variables.
- REPO_PATH is relative to the root of the client. REPO_OUTERPATH is not
  needed.
- REPO_INNERPATH is relative to the sub manifest root.
- REPO_OUTERPATH is the path for the sub manifest root relative to the
  root of the client.

Change-Id: I031692891cfef2634d1358584d27a6a4df735c20
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334899
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-04-14 14:31:47 +00:00
LaMont Jones
d56e2eb421 manifest_xml: use Superproject to hold XML content
Always create Superproject when there is a <superproject> tag, and have
it hold the XML content, similar to how other manifest elements are
handled.

This also adds SetQuiet and SetPrintMessages to Superproject
consistent with manifest.SetUseLocalManifests.

Change-Id: I522bf3da542006575799f0640c67f7052704f266
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334641
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-04-12 15:46:23 +00:00
Daniel Andersson
d52ca421d5 sync: respect sync-c manifest option
The documentation states that a `sync-c` attribute in the manifest file
can set a default for whether only the current branch should be fetched
or all branches. This seems to have been broken for some time.

Commit 7356114 introduced the `--no-current-branch` CLI option and
relied on getting `None` via `optparse` if neither `--current-branch`
nor `--no-current-branch` was set to distinguish it from a boolean
value. If `None` was received, it would read the value from the manifest
option `sync-c`. The parsing went through the utility function
`_GetCurrentBranchOnly` which returned `True` if `--current-branch` had
been given on the command-line, or fell back on the "superproject"
setting, which would either return `True` or `None`. This would
incorrectly make `repo` fall back to the manifest setting even if the
user had given `--no-current-branch` if no superproject was requested --
the manifest became "too powerful":

Command-line         Using superproject  → `current_branch_only`
------------         ------------------  -----------------------
                     No                  From manifest
                     Yes                 True
--current-branch     No                  True
--current-branch     Yes                 True
--no-current-branch  No                  From manifest ← wrong
--no-current-branch  Yes                 True

In commit 0cb6e92 the superproject configuration value reading changed
from something that could return `None` to something that always
returned a boolean. If it returned `False`, this would then incorrectly
make `repo` ignore the manifest option even if neither
`--current-branch` nor `--no-current-branch` had been given. The
manifest default became useless:

Command-line         Using superproject  → `current_branch_only`
------------         ------------------  -----------------------
                     No                  False ← wrong
                     Yes                 True
--current-branch     No                  True
--current-branch     Yes                 True
--no-current-branch  No                  False
--no-current-branch  Yes                 True

By swapping the order in which the command-line option target and the
superproject setting is evaluated, things should work as documented:

Command-line         Using superproject  → `current_branch_only`
------------         ------------------  -----------------------
                     No                  From manifest
                     Yes                 True
--current-branch     No                  True
--current-branch     Yes                 True
--no-current-branch  No                  False
--no-current-branch  Yes                 True

Change-Id: I933c232d2fbecc6b9bdc364ebac181798bce9175
Tested-by: Daniel Andersson <daniel.r.andersson@volvocars.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334270
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-08 21:06:37 +00:00
LaMont Jones
a2ff20dd20 manifest_xml: Add Load and Unload methods
- do not call the internal method from subcmds/sync.py.
- use the correct default groups for submanifests.
- only sync the superproject when we are told to.

Change-Id: I81e4025058f1ee564732b9e17aecc522f6b5f626
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334639
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2022-04-08 19:52:04 +00:00
LaMont Jones
55ee304304 Fix sub manifest handling
Also fixes some typos

Change-Id: Id2ba5834ba3a74ed3f29c36d2c0030737dc63e35
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334579
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-06 21:04:46 +00:00
LaMont Jones
409407a731 init: add multi-manifest support
This moves more of the manifest project handling into ManifestProject.

Change-Id: Iecdafbec18cccdfd8e625753c3bd1bcddf2b227f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334520
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-06 17:02:40 +00:00
LaMont Jones
d82be3e672 Move manifest config logic into ManifestProject
Use ManifestProject properties for config values.

Change-Id: Ib4ad90b0d9a089916e35615b8058942e6d01dc04
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334519
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-06 16:59:45 +00:00
LaMont Jones
9b03f15e8e project: add ManifestProject.Sync()
Move the logic to sync a ManifestProject out of subcmds/init.py

Change-Id: Ia9d00f3da1dc3c5dada84c4d19cf9802c2346cb0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334140
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-04-01 15:48:04 +00:00
LaMont Jones
cc879a97c3 Add multi-manifest support with <submanifest> element
To be addressed in another change:
 - a partial `repo sync` (with a list of projects/paths to sync)
   requires `--this-tree-only`.

Change-Id: I6c7400bf001540e9d7694fa70934f8f204cb5f57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322657
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-02-17 21:57:55 +00:00
XD Trol
630876f9e4 init: add an option --enable-git-lfs-filter
It was reported that git-lfs did not work with git-repo. Specifically,
`git read-tree -u` run by `repo sync` would fail git-lfs's smudge
filter. See https://github.com/github/git-lfs/issues/1422.

In fact, by the time `git read-tree -u` is run, the repository is not
bare. It is just that, the working directory is not the same as the
.git directory. git-lfs's filter should work. No one seems to have
delved into that issue.

Today, with newer versions of git-repo and git-lfs, that issue will
not reproduce. Tested with
- git 2.33, git-lfs 2.13 on macOS
- git 2.17, git-lfs 2.3 on ubuntu

So, it seems fine to add an option --enable-git-lfs-filter, default to
false, and stat that it may not work with older versions of git and
git-lfs in the help doc.

Bug: https://crbug.com/gerrit/14516
Change-Id: I8d21854eeeea541e072f63d6b10ad1253b1a9826
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328359
Tested-by: XD Trol <milestonejxd@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2022-01-26 01:47:20 +00:00
Mike Frysinger
4aa8584ec6 init: make bad --repo-rev settings more clear
If the user passes a bad --repo-rev setting in a new checkout, add a
tip to the error message that their option is probably bad instead of
just saying "unable to resolve".

If the user has already initialized a checkout, we'd display a raw
traceback which would confuse them.  Swallow that and also include
the --repo-rev tip.

Bug: https://crbug.com/gerrit/15610
Change-Id: I5d72513c7b37bf9bb5d19862fcdfaf0d1f44e886
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328820
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2022-01-25 18:54:42 +00:00
LaMont Jones
a535ae4418 branches: Fix "not in" handling
If the branch is current, or present in less than half of the projects,
list which projects it is *in*.

Otherwise, correctly detect which projects (by relpath) it is not in.

Previously, the "not in" path would incorrectly list all projects.

Change-Id: Ia153856f577035a51f538b7bf5d3135b70c69d52
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328199
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
2022-01-21 17:20:21 +00:00
Raman Tenneti
cf0ba48649 sync: With --mirror option, don't display no-use-superproject... message.
+ Display 'Defaulting to no-use-superproject because there is no working tree.'
  message if --use-superproject option is used and we are not using
  superproject because manifest is either a mirror or is an archive.

Tested the code with the following commands.

$ ./run_tests -v

Tested the sync code by using repo_dev alias and pointing to this CL.

$ repo init -u https://android.googlesource.com/mirror/manifest --mirror

$ repo_dev sync
Receiving objects: 100% (3/3), done.eiving objects:  33% (1/3)

$ repo_dev sync --use-superproject
Defaulting to no-use-superproject because there is no working tree.
Fetching:  0% (0/2158) warming up

Bug: https://crbug.com/gerrit/15368
Change-Id: I16b87ee9623315dbc3100b612b1decdaab7ac1dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/325797
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-12-07 16:46:41 +00:00
Raman Tenneti
4a478edb44 init, sync: fixed flake8 warnings.
Tested:
+ run_tests
+ flake8 subcmds/init.py
+ flake8 subcmds/sync.py

Change-Id: Ie337481d8a210bfc49b0745f75c05a308a0e74d3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/324155
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-18 16:22:40 +00:00
Raman Tenneti
6bd89aa657 superproject: Inherit --no-use-superproject with --mirror option.
init.py
+ Similar to opt.archive, gave an error if --mirror option is
  used with --use-superproject.

sync.py
+ Defaulted to --no-use-superproject if manifest is a mirror or
  archive (similar to error at line# 1067).

Tested:
+ run_tests
+ flake8 (will fix known errors in another CL).

$ repo_dev init -u sso://googleplex-android.git.corp.google.com/platform/manifest --use-superproject --mirror
Usage: repo init [options] [manifest url]

main.py: error: --mirror and --use-superproject cannot be used together.

+ repo init and repo sync with --mirror and without --mirror
  options.
  $ repo_dev init -u https://android.googlesource.com/platform/manifest
  $ repo_dev sync
    ...superproject.git: Initial setup for superproject completed.

+ With --mirror option, verfied there are no exceptions in git_superproject.py

Bug: [google internal] b/206537893
Change-Id: I059f20e76f0ab36f0587f29779bb53ede4663bd4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323955
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-18 01:27:41 +00:00
Mike Frysinger
fdeb20f43f sync: link the internal-fs-layout doc into checkouts
This should make it easy to discover for people poking around .repo/.

Change-Id: Ie5051551f25127c0592df5e36efba7bb2263e5d4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323701
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-15 01:39:53 +00:00
Mike Frysinger
3001d6a426 help: fix grammar in help text
Bug: https://crbug.com/gerrit/14838
Change-Id: Ic5000921ba9a1baa086153630ebbb429e3d17642
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323515
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-12 17:30:32 +00:00
Mike Frysinger
0531a623e1 sync: make --prune the default
If a remote deletes a ref, and it points to an object that doesn't
exist locally, we can get into a bad state, and the only way for the
user to recover is to run `repo sync --prune` (and to know that is
the option they need).  The error message is not helpful:

fatal: bad object refs/remotes/cros/firmware-zork-13421.B-master
error: https://chromium.googlesource.com/chromiumos/platform/ec did not send all necessary objects

This situation can also come up when the remote renames refs in a
UNIX FS incompatible way.  For example, replacing refs/heads/foo
with refs/heads/foo/bar.

Also add a --no-prune option for users to disable the behavior.

Bug: https://issuetracker.google.com/203366450
Change-Id: Icf45d838a10938feb091d29800f7e49240830ec3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322956
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-05 20:13:30 +00:00
Mike Frysinger
2273f46cb3 sync: fix --tags option
This has been broken since it was added where --tags was actually
the same as --no-tags.  Oddly, it was copied from init where the
logic is correct.

Bug: https://crbug.com/gerrit/12401
Change-Id: I15b89da1a655176a11bebc22573b25c728055328
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322955
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-11-05 20:13:02 +00:00
Raman Tenneti
6251729cb4 superproject: added 'implies -c' in the help of --use-superproject option.
sync.py: deleted unused import errno.

Tested:
$ ./run_tests
$ flake8 repo subcmds/sync.py

Bug: https://crbug.com/gerrit/15208
Change-Id: I2bb3098f5602ded3861e000100766041ad93b53d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322555
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-11-01 22:15:53 +00:00
Jack Neus
198838599c fetch: Fix stderr handling for gsutil
Previously gsutil stderr was getting piped into stdout, which
yields bad results if there are non-fatal warnings in stderr.

Additionally, we should fail outright if gsutil fails (by adding
`check = True`) rather than fail later on when we try to sync to
a manifest that is in fact just a stderr dump.

BUG=none
TEST=manual runs with bad gs urls

Change-Id: Id71791d0c3f180bd0601ef2c783a8e8e4afa8f59
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321935
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-26 22:18:28 +00:00
Jack Neus
03ff276cd7 sync: properly handle standalone manifests for sync command
sync should not attempt to sync the manifest project if it was
created from a standalone manifest. The current work around is to
run sync with --nmu.

BUG=none
TEST=manual runs

Change-Id: I2e121af0badf9642143e77c7af89d1c2d993b0f3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321195
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-15 17:20:00 +00:00
Allen Webb
4ee4a45d03 subcmds/sync: Use pack-refs instead of gc for redundant gitdirs.
Previously `git gc` was being run on every gitdir even when they shared
the same objects. Instead only call it once and use pack-refs for the
gitdirs that were not gc'ed.

Bug: https://crbug.com/gerrit/15113
Test: repo sync -j # and check that git pack-refs is called
Change-Id: Icff37ab3ec78cfb44391d8cc7f2d875991532320
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320275
Tested-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-14 12:27:12 +00:00
Jack Neus
0f6f16ed17 repo: more arg checking for --standalone-manifest re-inits
`repo init` doesn't do anything on re-init when the checkout has
been initialized using --standalone manifest. Rather than let the
tool run through its existing flows (which happen to noop), check
the args and explicitly quit if a bare `repo init` is run on a
standalone checkout.

BUG=none
TEST=manual tests

Change-Id: Ie4346ef6df1282ec3e3f8045a08138c93653fece
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320735
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-11 18:58:11 +00:00
Jack Neus
76491590b8 repo: fix bug with --standalone-manifest
We were accidentally always setting manifest.standlone in config,
which was messing up behavior for standard use cases.

BUG=gerrit:15160
TEST=manual runs

Change-Id: Ic80f084ae97de5721aced3bb52d3ea9115f8d833
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320715
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-11 18:57:57 +00:00
Allen Webb
669efd0fd7 subcmds/sync: Disable autoDetach for git gc.
gc.autoDetach is enabled by default which makes 'git gc --auto' return
immediately and run in background. This can lead to a pile up of
operations all using large amounts of memory at the same time. To avoid
this set gc.autoDetach to false so that the garbage collect task waits
for instances to finish before spawning more.

Bug: https://crbug.com/gerrit/15113
Test: repo sync -j # and check the number of 'git gc' processes
Change-Id: Ic0815156ba3db03972968f33f6f9f51e4928f23b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319835
Tested-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-10-05 14:12:01 +00:00
Mike Frysinger
9d96f58f5f make file removal a bit more robust
Some of the file removal calls are subject to race conditions (if
something else deletes the file), so extend our remove API to have
an option to ignore ENOENT errors.  Then update a bunch of random
call sites to use this new functionality.

Change-Id: I31a9090e135452033135337a202a4fc2dbf8b63c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319195
Reviewed-by: Sean McAllister <smcallis@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2021-09-28 16:06:50 +00:00
Jack Neus
c474c9cba1 repo: Add support for standalone manifests
Added --standalone_manifest to repo tool. If set, the
manifest is downloaded directly from the appropriate source
(currently, we only support GS) and used instead of creating
a manifest git checkout. The manifests.git repo is still created to
keep track of various config but is marked as being for a standalone
manifest so that the repo tool doesn't try to run networked git
commands in it.

BUG=b:192664812
TEST=existing tests (no coverage), manual runs

Change-Id: I84378cbc7f8e515eabeccdde9665efc8cd2a9d21
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312942
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2021-09-28 15:40:46 +00:00
Raman Tenneti
6448a4f2af sync: Log repo sync state events as 'data' events.
git_trace2_event_log.py:
+ Added LogDataConfigEvents method to log 'data' events.
  Sync's current_sync_state and previous_sync_state are logged
  as 'data' events in the current log.

  It logs are key/value in the |config| argument. Each key is
  prefixed with |prefix| argument.

  The following are sample events that are logged during repo sync.

   {"event":"data",
   "sid":"repo-20210914T181545Z-P000330c0/repo-20210914T181545Z-P000330c0",
   "thread":"MainThread",
   "time":"2021-09-14T18:16:19.935846Z",
   "key":"previous_sync_state/repo.syncstate.main.synctime",
   "value":"2021-09-14T17:27:11.573717Z"}

   {"event":"data",
   "sid":"repo-20210914T181545Z-P000330c0/repo-20210914T181545Z-P000330c0",
   "thread":"MainThread",
   "time":"2021-09-14T18:16:19.955546Z",
   "key":"current_sync_state/repo.syncstate.main.synctime",
   "value":"2021-09-14T18:16:19.935979Z"}

tests/test_git_trace2_event_log.py:
+ Added unit tests

sync.py:
+ Changed logging calls to LogDataConfigEvents.

Tested:
$ ./run_tests

Tested it by running the following command multiple times.
$ repo_dev sync -j 20
  repo sync has finished successfully

  Verified config data is looged in trace2 event logs.

Bug: [google internal] b/199758376
Change-Id: I75fd830e90c1811ec28510538c99a2632b104e85
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317823
Reviewed-by: Josh Steadmon <steadmon@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
2021-09-14 21:36:12 +00:00