sync: Added logging of repo sync state and config options for analysis.

git_config.py:
+ Added SyncAnalysisState class, which saves the following data
  into the config object.
  ++ sys.argv, options, superproject's logging data.
  ++ repo.*, branch.* and remote.* parameters from config object.
  ++ current time as synctime.
  ++ Version number of the object.
+ All the keys for the above data are prepended with 'repo.syncstate.'
+ Added GetSyncAnalysisStateData and UpdateSyncAnalysisState methods
  to GitConfig object to save/get the above data.

git_trace2_event_log.py:
+ Added LogConfigEvents method with code from DefParamRepoEvents
  to log events.

sync.py:
+ superproject_logging_data is a dictionary that collects all the
  superproject data that is to be logged as trace2 event.
+ Sync at the end logs the previously saved syncstate.* parameters
  as previous_sync_state. Then it calls config's UpdateSyncAnalysisState
  to save and log all the current options, superproject logged data.

docs/internal-fs-layout.md:
+ Added doc string explaining [repo.syncstate ...] sections of
  .repo/manifests.git/config file.

test_git_config.py:
+ Added unit test for the new methods of GitConfig object.

Tested:
$ ./run_tests

$ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest

Tested it by running the following command multiple times.
$ repo_dev sync -j 20
  repo sync has finished successfully

  Verified config file has [syncstate ...] data saved.

Bug: [google internal] b/188573450
Change-Id: I1f914ce50f3382111b72940ca56de7c41b53d460
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/313123
Tested-by: Raman Tenneti <rtenneti@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Xin Li <delphij@google.com>
This commit is contained in:
Raman Tenneti
2021-07-28 14:36:49 -07:00
parent ae86a46022
commit 7954de13b7
6 changed files with 152 additions and 9 deletions

View File

@ -282,7 +282,7 @@ later is required to fix a server side protocol bug.
"""Returns True if current-branch or use-superproject options are enabled."""
return opt.current_branch_only or git_superproject.UseSuperproject(opt, self.manifest)
def _UpdateProjectsRevisionId(self, opt, args, load_local_manifests):
def _UpdateProjectsRevisionId(self, opt, args, load_local_manifests, superproject_logging_data):
"""Update revisionId of every project with the SHA from superproject.
This function updates each project's revisionId with SHA from superproject.
@ -293,6 +293,7 @@ later is required to fix a server side protocol bug.
args: Arguments to pass to GetProjects. See the GetProjects
docstring for details.
load_local_manifests: Whether to load local manifests.
superproject_logging_data: A dictionary of superproject data that is to be logged.
Returns:
Returns path to the overriding manifest file instead of None.
@ -312,6 +313,7 @@ later is required to fix a server side protocol bug.
submodules_ok=opt.fetch_submodules)
update_result = superproject.UpdateProjectsRevisionId(all_projects)
manifest_path = update_result.manifest_path
superproject_logging_data['updatedrevisionid'] = bool(manifest_path)
if manifest_path:
self._ReloadManifest(manifest_path, load_local_manifests)
else:
@ -964,8 +966,14 @@ later is required to fix a server side protocol bug.
self._UpdateManifestProject(opt, mp, manifest_name)
load_local_manifests = not self.manifest.HasLocalManifests
if git_superproject.UseSuperproject(opt, self.manifest):
manifest_name = self._UpdateProjectsRevisionId(opt, args, load_local_manifests) or opt.manifest_name
use_superproject = git_superproject.UseSuperproject(opt, self.manifest)
superproject_logging_data = {
'superproject': use_superproject,
'haslocalmanifests': bool(self.manifest.HasLocalManifests),
}
if use_superproject:
manifest_name = self._UpdateProjectsRevisionId(
opt, args, load_local_manifests, superproject_logging_data) or opt.manifest_name
if self.gitc_manifest:
gitc_manifest_projects = self.GetProjects(args,
@ -1079,6 +1087,15 @@ later is required to fix a server side protocol bug.
file=sys.stderr)
sys.exit(1)
# Log the previous sync analysis state from the config.
self.git_event_log.LogConfigEvents(mp.config.GetSyncAnalysisStateData(),
'previous_sync_state')
# Update and log with the new sync analysis state.
mp.config.UpdateSyncAnalysisState(opt, superproject_logging_data)
self.git_event_log.LogConfigEvents(mp.config.GetSyncAnalysisStateData(),
'current_sync_state')
if not opt.quiet:
print('repo sync has finished successfully.')