Since --jobs defaults to 0, not None, we never pull the value out
of the manifest. Treat values of 0 and None the same to fix.
Bug: http://b/239712300
Bug: http://b/260908907
Change-Id: I9b1026682072366616825fd72f90bd90c10a252f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354254
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
If syncing in _FetchOne fails with GitError, sync_result does not get
set. There's already a separate local variable for success; do the same
for remote_fetched instead of referring to the conditionally defined
named tuple.
This bug is originally caused by a combination of ad8aa697 "sync: only
print error.GitError, don't raise that exception." and 1eddca84 "sync:
use namedtuples for internal return values".
Change-Id: I0f9dbafb97f8268044e5a56a6f92cf29bc23ca6a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354176
Tested-by: Karsten Tausche <karsten@fairphone.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
The wrong revision of the change was submitted as
d793553804.
Change-Id: I6f3e4993cf40c30ccf0d69020177db8fe5f76b8c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353934
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Do not use a progress bar when not doing GC, and restrict activity in
that case to only repairing preciousObject state.
This also includes additional cleanup based on review comments from
previous changes.
Change-Id: I48581c9d25da358bc7ae15f40e98d55bec142331
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353514
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
This bug-cacher related code is no longer needed.
This reverts commit 891e8f72ce.
Change-Id: Ia94a2690ff149427fdcafacd39f5008cd60827d5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353774
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
REPO_AUTO_GC was introduced as a way for users to restore the previous
default behavior, since the default changed at the same time as the
option was added. As such, it should be marked as deprecated, and
removed entirely in a future release.
Change-Id: Ib73d98fbea693e7057cc4587928c225a9e4beab2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/353734
Reviewed-by: Sam Saccone <samccone@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Correct usage of project.relpath for multi manifest workspaces.
Change-Id: Idc32873552fcdae6eec7b03dde2b2f31134b72fd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/347534
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
Add an environment variable to restore previous behavior, since the
older version of repo does not support `--auto-gc`.
Change-Id: I874dfb8fc3533a97b8adfd52125eb3d1d75e2f3c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/351194
Reviewed-by: Sam Saccone <samccone@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
Adds --auto-gc and --no-auto-gc (default) options to control sync's
behavior around calling `git gc`.
Bug: b/184882274
Change-Id: I4d6ca3b233d79566f27e876ab2d79f238ebc12a9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/344535
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
If this is a project that is not using object sharing (there is only one
copy of the remote project) then clear preciousObjects.
To override this for a project, run:
git config --replace-all repo.preservePreciousObjects true
Change-Id: If3ea061c631c5ecd44ead84f68576012e2c7405c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/350235
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
When repo sync fails, if the workspace is a mirror, an uninitialized
variable is referenced.
Bug: crbug.com/gerrit/16356
Change-Id: I1dba9f92319b9cbfd18460327560a395c88a089f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/349654
Reviewed-by: Sam Saccone <samccone@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
Use pre-3.9 syntax for NamedTuple, so that users do not need to have
python 3.9 or later installed.
Bug: b/255632143, crbug.com/gerrit/16355
Test: manually verified with python 3.8
Change-Id: I488d2d5267ed98d5c55c233cc789e629f1911c9d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/349395
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
Replace tuple returns with namedtuples, to simplify adding new fields.
Extend the Sync_NetworkHalf return value to:
- success: True if successful (the former return value)
- remote_fetched: True if we called `git fetch`
Change-Id: If63c24c2f849523f77fa19c05bbf23a5e9a20ba9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/344534
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
This is ENABLED BY DEFAULT due to data corruption potential. To disable
it, set REPO_BACKUP_OBJECTS=0 in the environment.
While the workspace will grow over time, this provides a recovery path
for an issue where objects are erroneously deleted from the workspace,
resulting in lost work. Once the root cause is determined, we will be
able to stop saving backups again.
Backups are kept in .git/objects/.repo/pack.bak
Bug: https://crbug.com/gerrit/16247
Change-Id: Ib8b5c9b4bf0dfa9e29606e0f5c881d65996b2a40
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/345114
Reviewed-by: Jonathan Tan <jonathantanmy@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Xin Li <delphij@google.com>
If the server responds with an HTML page, we should show that to the
user instead of crashing with XML errors.
Bug: https://crbug.com/gerrit/15936
Change-Id: I52e6b781c3bb6a6c9f6ecbe2e0907044876cdc8d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337519
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
For projects that no longer share their per-project objects directly, we
no longer have to disable the git settings that disable pruning. See
commit "project: stop directly sharing objects/ between shared projects"
for more details.
Bug: https://crbug.com/gerrit/15553
Change-Id: Ica0e83c3002716424c2bc9111b3b3d3a76c30973
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/337535
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
For --jobs-network, the logic is now:
* If the user specifies --jobs-network, use that.
* Else, if the user specifies --jobs, use that.
* Else, if the manifest specifies sync-j, use that.
* Else, default to 1.
Then we limit the jobs count based on the softlimit RLIMIT_NOFILE.
For --jobs-checkout, the logic is now:
* If the user specifies --jobs-checkout, use that.
* Else, if the user specifies --jobs, use that.
* Else, if the manifest specifies sync-j, use that.
* Else, default to DEFAULT_LOCAL_JOBS which is based on user's ncpus.
Then we limit the jobs count based on the softlimit RLIMIT_NOFILE.
For garbage collecting, the logic is now:
* If the user specifies --jobs, use that.
* Else, if the manifest specifies sync-j, use that.
* Else, default to the user's ncpus.
Then we limit the jobs count based on the softlimit RLIMIT_NOFILE.
Having to factor in the manifest settings makes this more complicated
which is why we delay processing of defaults until after we've synced
the manifest projects.
Bug: http://b/239712300
Change-Id: Id27cda63c76c156f1d63f6a20cb2c4ceeb3d547c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/341394
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: LaMont Jones <lamontjones@google.com>
If the user says `--no-use-superproject`, then do not bother syncing the
superproject.
Also add/update docstrings and comments throughout.
Change-Id: I9cdad706130501bab9a22d3099a1dae605e9c194
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/338975
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
With this change, partial syncs (sync with a project list) are again
supported.
If the updated manifest includes new sub manifests, download them
inheriting options from the parent manifestProject.
Change-Id: Id952f85df2e26d34e38b251973be26434443ff56
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334819
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
We need to iterate over multiple manifests, and generally use the
outer_client.manifest for multi-manifest support. This refactors the
use of self.manifest into a chosen manifest.
Change-Id: I992f21d610c929675e99555ece9c38df4b635839
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334699
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Make git_superproject independent of the command line by passing
the specific value instead of requiring the caller to have an
optparse.Values object to pass in.
Flag --use-superproject and --archive as incompatible in subcmds/init.py
Change-Id: Ied7c874b312e151038df903c8af4328f070f387c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/335135
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Always create Superproject when there is a <superproject> tag, and have
it hold the XML content, similar to how other manifest elements are
handled.
This also adds SetQuiet and SetPrintMessages to Superproject
consistent with manifest.SetUseLocalManifests.
Change-Id: I522bf3da542006575799f0640c67f7052704f266
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334641
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
The documentation states that a `sync-c` attribute in the manifest file
can set a default for whether only the current branch should be fetched
or all branches. This seems to have been broken for some time.
Commit 7356114 introduced the `--no-current-branch` CLI option and
relied on getting `None` via `optparse` if neither `--current-branch`
nor `--no-current-branch` was set to distinguish it from a boolean
value. If `None` was received, it would read the value from the manifest
option `sync-c`. The parsing went through the utility function
`_GetCurrentBranchOnly` which returned `True` if `--current-branch` had
been given on the command-line, or fell back on the "superproject"
setting, which would either return `True` or `None`. This would
incorrectly make `repo` fall back to the manifest setting even if the
user had given `--no-current-branch` if no superproject was requested --
the manifest became "too powerful":
Command-line Using superproject → `current_branch_only`
------------ ------------------ -----------------------
No From manifest
Yes True
--current-branch No True
--current-branch Yes True
--no-current-branch No From manifest ← wrong
--no-current-branch Yes True
In commit 0cb6e92 the superproject configuration value reading changed
from something that could return `None` to something that always
returned a boolean. If it returned `False`, this would then incorrectly
make `repo` ignore the manifest option even if neither
`--current-branch` nor `--no-current-branch` had been given. The
manifest default became useless:
Command-line Using superproject → `current_branch_only`
------------ ------------------ -----------------------
No False ← wrong
Yes True
--current-branch No True
--current-branch Yes True
--no-current-branch No False
--no-current-branch Yes True
By swapping the order in which the command-line option target and the
superproject setting is evaluated, things should work as documented:
Command-line Using superproject → `current_branch_only`
------------ ------------------ -----------------------
No From manifest
Yes True
--current-branch No True
--current-branch Yes True
--no-current-branch No False
--no-current-branch Yes True
Change-Id: I933c232d2fbecc6b9bdc364ebac181798bce9175
Tested-by: Daniel Andersson <daniel.r.andersson@volvocars.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334270
Reviewed-by: Mike Frysinger <vapier@google.com>
- do not call the internal method from subcmds/sync.py.
- use the correct default groups for submanifests.
- only sync the superproject when we are told to.
Change-Id: I81e4025058f1ee564732b9e17aecc522f6b5f626
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334639
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
Use ManifestProject properties for config values.
Change-Id: Ib4ad90b0d9a089916e35615b8058942e6d01dc04
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/334519
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
To be addressed in another change:
- a partial `repo sync` (with a list of projects/paths to sync)
requires `--this-tree-only`.
Change-Id: I6c7400bf001540e9d7694fa70934f8f204cb5f57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322657
Tested-by: LaMont Jones <lamontjones@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
+ Display 'Defaulting to no-use-superproject because there is no working tree.'
message if --use-superproject option is used and we are not using
superproject because manifest is either a mirror or is an archive.
Tested the code with the following commands.
$ ./run_tests -v
Tested the sync code by using repo_dev alias and pointing to this CL.
$ repo init -u https://android.googlesource.com/mirror/manifest --mirror
$ repo_dev sync
Receiving objects: 100% (3/3), done.eiving objects: 33% (1/3)
$ repo_dev sync --use-superproject
Defaulting to no-use-superproject because there is no working tree.
Fetching: 0% (0/2158) warming up
Bug: https://crbug.com/gerrit/15368
Change-Id: I16b87ee9623315dbc3100b612b1decdaab7ac1dc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/325797
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
init.py
+ Similar to opt.archive, gave an error if --mirror option is
used with --use-superproject.
sync.py
+ Defaulted to --no-use-superproject if manifest is a mirror or
archive (similar to error at line# 1067).
Tested:
+ run_tests
+ flake8 (will fix known errors in another CL).
$ repo_dev init -u sso://googleplex-android.git.corp.google.com/platform/manifest --use-superproject --mirror
Usage: repo init [options] [manifest url]
main.py: error: --mirror and --use-superproject cannot be used together.
+ repo init and repo sync with --mirror and without --mirror
options.
$ repo_dev init -u https://android.googlesource.com/platform/manifest
$ repo_dev sync
...superproject.git: Initial setup for superproject completed.
+ With --mirror option, verfied there are no exceptions in git_superproject.py
Bug: [google internal] b/206537893
Change-Id: I059f20e76f0ab36f0587f29779bb53ede4663bd4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323955
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
This should make it easy to discover for people poking around .repo/.
Change-Id: Ie5051551f25127c0592df5e36efba7bb2263e5d4
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/323701
Reviewed-by: Jack Neus <jackneus@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
If a remote deletes a ref, and it points to an object that doesn't
exist locally, we can get into a bad state, and the only way for the
user to recover is to run `repo sync --prune` (and to know that is
the option they need). The error message is not helpful:
fatal: bad object refs/remotes/cros/firmware-zork-13421.B-master
error: https://chromium.googlesource.com/chromiumos/platform/ec did not send all necessary objects
This situation can also come up when the remote renames refs in a
UNIX FS incompatible way. For example, replacing refs/heads/foo
with refs/heads/foo/bar.
Also add a --no-prune option for users to disable the behavior.
Bug: https://issuetracker.google.com/203366450
Change-Id: Icf45d838a10938feb091d29800f7e49240830ec3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322956
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
This has been broken since it was added where --tags was actually
the same as --no-tags. Oddly, it was copied from init where the
logic is correct.
Bug: https://crbug.com/gerrit/12401
Change-Id: I15b89da1a655176a11bebc22573b25c728055328
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322955
Reviewed-by: Andrew Lamb <andrewlamb@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
sync should not attempt to sync the manifest project if it was
created from a standalone manifest. The current work around is to
run sync with --nmu.
BUG=none
TEST=manual runs
Change-Id: I2e121af0badf9642143e77c7af89d1c2d993b0f3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/321195
Tested-by: Jack Neus <jackneus@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Previously `git gc` was being run on every gitdir even when they shared
the same objects. Instead only call it once and use pack-refs for the
gitdirs that were not gc'ed.
Bug: https://crbug.com/gerrit/15113
Test: repo sync -j # and check that git pack-refs is called
Change-Id: Icff37ab3ec78cfb44391d8cc7f2d875991532320
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/320275
Tested-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
gc.autoDetach is enabled by default which makes 'git gc --auto' return
immediately and run in background. This can lead to a pile up of
operations all using large amounts of memory at the same time. To avoid
this set gc.autoDetach to false so that the garbage collect task waits
for instances to finish before spawning more.
Bug: https://crbug.com/gerrit/15113
Test: repo sync -j # and check the number of 'git gc' processes
Change-Id: Ic0815156ba3db03972968f33f6f9f51e4928f23b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319835
Tested-by: Allen Webb <allenwebb@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Some of the file removal calls are subject to race conditions (if
something else deletes the file), so extend our remove API to have
an option to ignore ENOENT errors. Then update a bunch of random
call sites to use this new functionality.
Change-Id: I31a9090e135452033135337a202a4fc2dbf8b63c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319195
Reviewed-by: Sean McAllister <smcallis@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
git_trace2_event_log.py:
+ Added LogDataConfigEvents method to log 'data' events.
Sync's current_sync_state and previous_sync_state are logged
as 'data' events in the current log.
It logs are key/value in the |config| argument. Each key is
prefixed with |prefix| argument.
The following are sample events that are logged during repo sync.
{"event":"data",
"sid":"repo-20210914T181545Z-P000330c0/repo-20210914T181545Z-P000330c0",
"thread":"MainThread",
"time":"2021-09-14T18:16:19.935846Z",
"key":"previous_sync_state/repo.syncstate.main.synctime",
"value":"2021-09-14T17:27:11.573717Z"}
{"event":"data",
"sid":"repo-20210914T181545Z-P000330c0/repo-20210914T181545Z-P000330c0",
"thread":"MainThread",
"time":"2021-09-14T18:16:19.955546Z",
"key":"current_sync_state/repo.syncstate.main.synctime",
"value":"2021-09-14T18:16:19.935979Z"}
tests/test_git_trace2_event_log.py:
+ Added unit tests
sync.py:
+ Changed logging calls to LogDataConfigEvents.
Tested:
$ ./run_tests
Tested it by running the following command multiple times.
$ repo_dev sync -j 20
repo sync has finished successfully
Verified config data is looged in trace2 event logs.
Bug: [google internal] b/199758376
Change-Id: I75fd830e90c1811ec28510538c99a2632b104e85
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/317823
Reviewed-by: Josh Steadmon <steadmon@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
1) If the manifest has superproject tag (git_master, etc), then
display error/warning messages (as it is doing today)
2) If the manifest doesn't have superproject tag (nest, chromeos
manifests), then don't display any error/warning messages about
superrproject (behave as though user has specified
--no-use-superproject).
3) Print error/warning messages if --use-superproject passed as
argument to repo sync.
4) No change in behavior for the repo init command.
git_superproject.py:
+ Fixed typo in _WriteManifestFile method name
+ Superproject accepts print_message as an argument and it defaults
to True. All messages that are printed to stderr are controlled by
this flag. If it is True, then messages get printed.
+ Added PrintMessages function which return true if either
--use-superproject is specified on the command line or if the
manifest has a superproject tag.
sync.py:
+ Displays the warning message if PrintMessgages are enabled and
passes that as argument to superproject object.
+ Added 'hassuperprojecttag' trace2 log entry for analysis. We can
find users/branches that are using superproject, but the manifest is
missing the superproject tag.
Tested:
$ ./run_tests
+ Verified printing of messages with and without superproject tag, with
with --use-superproject option.
+ aosp-master
$ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest
$ repo_dev sync
+ A manifest without superproject tag.
$ repo_dev init -m $(pwd)/manifest_7482982.xml
$ repo_dev sync -n -c -j32 -m $(pwd)/manifest_7482982.xml
Bug: [google internal] b/196411099
Change-Id: I92166dcad15a4129fab82edcf869e7c8db3efd4b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/314982
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
git_config.py:
+ Added SyncAnalysisState class, which saves the following data
into the config object.
++ sys.argv, options, superproject's logging data.
++ repo.*, branch.* and remote.* parameters from config object.
++ current time as synctime.
++ Version number of the object.
+ All the keys for the above data are prepended with 'repo.syncstate.'
+ Added GetSyncAnalysisStateData and UpdateSyncAnalysisState methods
to GitConfig object to save/get the above data.
git_trace2_event_log.py:
+ Added LogConfigEvents method with code from DefParamRepoEvents
to log events.
sync.py:
+ superproject_logging_data is a dictionary that collects all the
superproject data that is to be logged as trace2 event.
+ Sync at the end logs the previously saved syncstate.* parameters
as previous_sync_state. Then it calls config's UpdateSyncAnalysisState
to save and log all the current options, superproject logged data.
docs/internal-fs-layout.md:
+ Added doc string explaining [repo.syncstate ...] sections of
.repo/manifests.git/config file.
test_git_config.py:
+ Added unit test for the new methods of GitConfig object.
Tested:
$ ./run_tests
$ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest
Tested it by running the following command multiple times.
$ repo_dev sync -j 20
repo sync has finished successfully
Verified config file has [syncstate ...] data saved.
Bug: [google internal] b/188573450
Change-Id: I1f914ce50f3382111b72940ca56de7c41b53d460
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/313123
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Skip updating the superproject when -l is present and use the existing
superproject, if available (this would make sync -l work as it's
intended to do), and fall back to sync without superproject when not
(this would catch the case when superproject is enabled by automatic
rollout).
Tested:
$ repo sync -j 20 -n
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
/usr/local/google/home/rtenneti/work/android/src/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
Fetching: 100% (1032/1032), done in 41.184s
...
$ repo_dev sync -j 20 -l
prebuilts/asuite/: discarding 1 commits
prebuilts/runtime/: discarding 1 commits
...
repo sync has finished successfully.
+ With superproject-override.xml and test it.
$ ls -l .repo/exp-superproject/
total 176
drwxr-xr-x 7 rtenneti primarygroup 4096 Jul 27 14:10 925043f706ba64db713e9bf3b55987e2-superproject.git
-rw-r--r-- 1 rtenneti primarygroup 172742 Jul 27 14:10 superproject_override.xml
rtenneti@rtenneti:~/work/android/src/aosp$ repo_dev sync -j 20 -l
...
repo sync has finished successfully.
+ Rename the file superproject-override.xml and test it.
$ ls -l .repo/exp-superproject/
total 176
drwxr-xr-x 7 rtenneti primarygroup 4096 Jul 27 14:10 925043f706ba64db713e9bf3b55987e2-superproject.git
-rw-r--r-- 1 rtenneti primarygroup 172742 Jul 27 14:10 temp.xml
$ repo_dev sync -j 20 -l
Checking out: 1% (12/1031) platform/external/rust/crates/fallible-streaming-iteexternal/linux-kselftest/: discarding 1 commits
prebuilts/remoteexecution-client/: discarding 1 commits
Checking out: 51% (536/1031) platform/prebuilts/gcc/darwin-x86/aarch64/....
....
Checking out: 100% (1031/1031), done in 5.478s
repo sync has finished successfully.
Bug: [google internal] b/184368268
Change-Id: I3aba5872e4f7c299977b92c2a39847ef28698c5a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312962
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
This is similar to smart sync, allowing sync to benefit from the patched
manifest.
Bug: [google internal] b/190688390
Change-Id: I158a80afceca606dcd81ec76b2caede369f7ed03
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312142
Reviewed-by: Raman Tenneti <rtenneti@google.com>
Tested-by: Xin Li <delphij@google.com>
superproject is going to be default for some users. This change
doesn't fail for repo init or repo sync if source couldn't be synced
because of errors in superproject and superproject=true in the config
file. The commands will fail if --use-superproject is specified on
the command line explicitly.
The error messages are logged with trace2 event logs and will be
monitored.
+ sync - When there are errors with superproject and git_superproject
says it is fatal failure, sync will exit only when --use-superproject
option is specified on the command line.
+ init - command doesn't fail *if there are any superproject errors),
but it will print a warning message and logs message via trace2 event
logs. For fatal errors, init will exit only when --use-superproject
option is specified on the command line.
+ All git commands log the command that is being executed so trace2
event logs will know the manifest, remote url and the branch name.
There is no functional change other than fatal errors are honored with
--use-supeproject option with init/sync commands.
Tested the code with the following commands.
$ ./run_tests -v
Test 1 - sync'ing without errors
--------------------------------
Added the following lines to '~/.repoconfig/config
[repo]
superproject = true
$ repo_dev init -u https://android.googlesource.com/platform/manifest -b android-s-beta-2
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
repo: error: git fetch call failed, command: git ['fetch', 'https://android.googlesource.com/platform/superproject', '--depth', '1', '--force', '--no-tags', '--filter', 'blob:none', 'android-s-beta-2:android-s-beta-2'], return code: 128, stderr: fatal: couldn't find remote ref android-s-beta-2
warning: git update of superproject failed, repo sync will not use superproject to fetch source; while this error is not fatal, and you can continue to run repo sync, please run repo init with the --no-use-superproject option to stop seeing this warning
Your identity is: Raman Tenneti <rtenneti@google.com>
If you want to change this, please re-run 'repo init' with --config-name
repo has been initialized in /usr/local/google/home/rtenneti/work/drive2/android/test
$ repo_dev sync
remote: Total 4 (delta 1), reused 4 (delta 1)
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
/usr/local/google/home/rtenneti/work/drive2/android/aosp/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Initial setup for superproject completed.
...
Test 2 - init and sync fail when --use-superproject option is passed
--------------------------------------------------------------------
$ repo_dev init -u https://android.googlesource.com/platform/manifest -b android-s-beta-2 --use-superproject
remote: Total 57 (delta 16), reused 56 (delta 16)
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
repo: error: git fetch call failed, command: git ['fetch', 'https://android.googlesource.com/platform/superproject', '--depth', '1', '--force', '--no-tags', '--filter', 'blob:none', 'android-s-beta-2:android-s-beta-2'], return code: 128, stderr: fatal: couldn't find remote ref android-s-beta-2
warning: git update of superproject failed, repo sync will not use superproject to fetch source; while this error is not fatal, and you can continue to run repo sync, please run repo init with the --no-use-superproject option to stop seeing this warning
rtenneti@rtenneti2:~/work/drive2/android/test$ repo_dev sync --use-superproject
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
repo: error: git fetch call failed, command: git ['fetch', 'https://android.googlesource.com/platform/superproject', '--depth', '1', '--force', '--no-tags', '--filter', 'blob:none', 'android-s-beta-2:android-s-beta-2'], return code: 128, stderr: fatal: couldn't find remote ref android-s-beta-2
warning: Cannot get project commit ids from manifest
warning: Update of revisionId from superproject has failed, repo sync will not use superproject to fetch the source. Please resync with the --no-use-superproject option to avoid this repo warning.
Test 3 - git fetch command fails and git command is printed
-----------------------------------------------------------
With config change
$ repo_dev init -u https://android.googlesource.com/platform/manifest -b android-s-beta-2
...
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
.../android/test/.repo/exp-superproject/925043f706ba64db713e9bf3b55987e2-superproject.git: Performing initial setup for superproject; this might take several minutes.
repo: error: git fetch call failed,command: git ['fetch', 'https://android.googlesource.com/platform/superproject', '--depth', '1', '--force', '--no-tags', '--filter', 'blob:none', 'android-s-beta-2:android-s-beta-2'], return code: 128, stderr: fatal: couldn't find remote ref android-s-beta-2
warning: git update of superproject failed, repo sync will not use superproject to fetch source; while this error is not fatal and you can continue to run repo sync please run repo init with the --no-use-superproject option to avoid the repo warning
Your identity is: Raman Tenneti <rtenneti@google.com>
If you want to change this, please re-run 'repo init' with --config-name
repo has been initialized in ....
Test 4 - no superproject tag
-----------------------------
$ repo_dev init -u https://android.googlesource.com/platform/manifest -b pie-dev
...
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
repo error: superproject tag is not defined in manifest: .../android/pie_dev/.repo/manifest.xml
warning: git update of superproject failed, repo sync will not use superproject to fetch source; while this error is not fatal and you can continue to run repo sync please run repo init with the --no-use-superproject option to avoid the repo warning
Your identity is: Raman Tenneti <rtenneti@google.com>
If you want to change this, please re-run 'repo init' with --config-name
repo has been initialized in ...
$ repo_dev sync
NOTICE: --use-superproject is in beta; report any issues to the address described in `repo version`
repo error: superproject tag is not defined in manifest: /usr/local/google/home/rtenneti/work/drive2/android/pie_dev/.repo/manifest.xml
warning: Cannot get project commit ids from manifest
warning: Update of revsionId from superproject has failed. Please resync with --no-use-superproject option to avoid the repo warning.
Bug: [google internal] b/192614798
Bug: [google internal] b/Bug: [google internal] b/192614798
Change-Id: I9a97a0e7d9e609fad151bd7dd9cfc523eaa887cd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/311502
Reviewed-by: Amith Dsouza <amithds@google.com>
Reviewed-by: Xin Li <delphij@google.com>
Tested-by: Raman Tenneti <rtenneti@google.com>
When loading of superproject failed, we were resetting the manifest to
None, and later code would reload the manifest to see if there are
submodules, which would load the non-local manifest, causing sync with
superproject to fail.
Address this by setting the manifest_name to opt.manifest_name instead.
Bug: [google internal] b/189139268
Change-Id: I3616512e1c4b73e7eca0d83fd1fc474b825adbbf
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/311102
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Xin Li <delphij@google.com>
Repo will remember a choice and an expiration time of the choice, per
user, about whether to use superproject by default. When not specified
from command line and the choice is not expired, repo would use the
user default value.
When a user default value is not present and when the system wide
enable default is provided in git's system configuration, repo would
ask the user for a confirmation which will be valid for two weeks.
git_config.py: Add support for system config. When reading system
config, we would use --system to avoid hardcoding a path as the
value may be different on some other distributions.
git_superproject.py: Add a new subroutine, _UseSuperproject(), which
returns whether superproject should be used and whether it
is from a user configuration.
The value is determined in the following order:
1. If the user specifies either --use-superproject or
--no-use-superproject, then that choice is being used.
2. If neither is specified, we would then check the saved value
(upon repo init) and use that choice when there was a choice.
3. We then check if there is a saved and unexpired value for
user's choice in their ~/.gitconfig, and use the unexpired
choice, if available.
4. Finally, if all the above didn't give us a decision, and if
the git system configuration is providing a rollout hint, present
a prompt to user for their decision and save it in ~/.gitconfig.
subcmds/sync.py: Make use of the new UseSuperproject() provided by
git_superproject.py.
While there also silent stderr from git describe when determining the
version of repo.
Bug: [google internal] b/190688390
Change-Id: Iad3ee03026342ee500e5d65e2f0fa600d7637613
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/309762
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Xin Li <delphij@google.com>
