android/git-repo
1
Fork 0
mirror of https://gerrit.googlesource.com/git-repo synced 2024-12-27 07:16:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
v2.39
git-repo/docs/internal-fs-layout.md

277 lines
14 KiB
Markdown
Raw Permalink Normal View History

docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
# Repo internal filesystem layout
A reference to the `.repo/` tree in repo client checkouts.
Hopefully it's complete & up-to-date, but who knows!
*** note
**Warning**:
This is meant for developers of the repo project itself as a quick reference.
**Nothing** in here must be construed as ABI, or that repo itself will never
change its internals in backwards incompatible ways.
***
[TOC]
## .repo/ layout
All content under `.repo/` is managed by `repo` itself with few exceptions.
In general, you should not make manual changes in here.
If a setting was initialized using an option to `repo init`, you should use that
command to change the setting later on.
It is always safe to re-run `repo init` in existing repo client checkouts.
For example, if you want to change the manifest branch, you can simply run
`repo init --manifest-branch=<new name>` and repo will take care of the rest.
git_config: add support for repo-specific settings This allows people to write ~/.repoconfig/config akin to ~/.gitconfig and .repo/config akin to .git/config. This allows us to add settings specific to repo without mixing up git, and to persist in general. Change-Id: I1c6fbe31e63fb8ce26aa85335349c6ae5b1712c6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255832 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
2020-02-19 02:31:51 +00:00
* `config`: Per-repo client checkout settings using [git-config] file format.
* `.repo_config.json`: JSON cache of the `config` file for repo to
read/process quickly.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
### repo/ state
* `repo/`: A git checkout of the repo project. This is how `repo` re-execs
itself to get the latest released version.
It tracks the git repository at `REPO_URL` using the `REPO_REV` branch.
Those are specified at `repo init` time using the `--repo-url=<REPO_URL>`
init: rename --repo-branch to --repo-rev We refer to this as "revision" in help text, and in REPO_REV envvar, so rename to --repo-rev to be consistent. We keep --repo-branch for backwards compatibility, but as a hidden option. Bug: https://crbug.com/gerrit/11045 Change-Id: I1ecc282fba32917ed78a63850360c08469db849a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259352 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-03-14 18:35:26 +00:00
and `--repo-rev=<REPO_REV>` options.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
Any changes made to this directory will usually be automatically discarded
by repo itself when it checks for updates. If you want to update to the
latest version of repo, use `repo selfupdate` instead. If you want to
change the git URL/branch that this tracks, re-run `repo init` with the new
settings.
docs: Document .repo_localsyncstate.json Update docs to reflect the new internal filesystem layout. Bug: b/286126621, b/271507654 Change-Id: I8a2f8f36dff75544f32356ac5e31668f32ddffb3 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/383074 Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com> Tested-by: Gavin Mak <gavinmak@google.com>
2023-08-16 22:06:25 +00:00
* `.repo_fetchtimes.json`: Used by `repo sync` to record fetch times when
syncing the various projects.
* `.repo_localsyncstate.json`: Used by `repo sync` to detect and warn on
on partial tree syncs. Partial syncs are allowed by `repo` itself, but are
unsupported by many projects where `repo` is used.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
### Manifests
For more documentation on the manifest format, including the local_manifests
support, see the [manifest-format.md] file.
Add multi-manifest support with <submanifest> element To be addressed in another change: - a partial `repo sync` (with a list of projects/paths to sync) requires `--this-tree-only`. Change-Id: I6c7400bf001540e9d7694fa70934f8f204cb5f57 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/322657 Tested-by: LaMont Jones <lamontjones@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
2021-11-18 22:40:18 +00:00
* `submanifests/{submanifest.path}/`: The path prefix to the manifest state of
a submanifest included in a multi-manifest checkout. The outermost manifest
manifest state is found adjacent to `submanifests/`.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
* `manifests/`: A git checkout of the manifest project. Its `.git/` state
points to the `manifest.git` bare checkout (see below). It tracks the git
branch specified at `repo init` time via `--manifest-branch`.
The local branch name is always `default` regardless of the remote tracking
branch. Do not get confused if the remote branch is not `default`, or if
there is a remote `default` that is completely different!
No manual changes should be made in here as it will just confuse repo and
it won't automatically recover causing no new changes to be picked up.
* `manifests.git/`: A bare checkout of the manifest project. It tracks the
git repository specified at `repo init` time via `--manifest-url`.
No manual changes should be made in here as it will just confuse repo.
If you want to switch the tracking settings, re-run `repo init` with the
new settings.
manifest_xml: change .repo/manifest.xml to a plain file Changing this to a file instead of using a symlink serves two purposes: * We can insert some comments & doc links to help users learn what this is for, discover relevant documentation, and to discourage them from modifying things. * Windows requires Administrator access to use symlinks. With this last change, Windows users can get repo client checkouts with the new --worktree option and not need symlinks anywhere at all. Which means they no longer need to be an Administrator in order to `repo sync`. Change-Id: I9bc46824fd8d4b0f446ba84bd764994ca1e597e2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256313 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-21 05:49:41 +00:00
* `manifest.xml`: The manifest that repo uses. It is generated at `repo init`
and uses the `--manifest-name` to determine what manifest file to load next
out of `manifests/`.
Do not try to modify this to load other manifests as it will confuse repo.
If you want to switch manifest files, re-run `repo init` with the new
setting.
Older versions of repo managed this with symlinks.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
* `manifest.xml -> manifests/<manifest-name>.xml`: A symlink to the manifest
that the user wishes to sync. It is specified at `repo init` time via
`--manifest-name`.
* `manifests.git/.repo_config.json`: JSON cache of the `manifests.git/config`
file for repo to read/process quickly.
* `local_manifest.xml` (*Deprecated*): User-authored tweaks to the manifest
used to sync. See [local manifests] for more details.
* `local_manifests/`: Directory of user-authored manifest fragments to tweak
the manifest used to sync. See [local manifests] for more details.
### Project objects
docs: add warnings about repos data model For people coming across these docs and thinking that repo's methods are good to replicate, add a note warning them against doing so. Change-Id: I443a783794313851a6e7ba1c39baebac988bff9a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298164 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2021-02-25 08:13:31 +00:00
*** note
**Warning**: Please do not use repo's approach to projects/ & project-objects/
layouts as a model for other tools to implement similar approaches.
It has a number of known downsides like:
* [Symlinks do not work well under Windows](./windows.md).
* Git sometimes replaces symlinks under .git/ with real files (under unknown
circumstances), and then the internal state gets out of sync, and data loss
may ensue.
* When sharing project-objects between multiple project checkouts, Git might
automatically run `gc` or `prune` which may lead to data loss or corruption
(since those operate on leaf projects and miss refs in other leaves). See
https://gerrit-review.googlesource.com/c/git-repo/+/254392 for more details.
Instead, you should use standard Git workflows like [git worktree] or
[gitsubmodules] with [superprojects].
***
Update copyfile and linkfile if manifest updated Currently, copyfiles and linkfiles which marked by "<copyfile/>" and "<linkfile/>" in manifest will be created by first exec 'repo sync'. But if some "<copyfile/>" or "<linkfile/>" are removed in manifest, then 'repo sync', these removed item dest can not be removed in the sourcecode workspace. This patch is intent to fix this issue, by save a 'copy-link-files.json' in .repo and then compared with new dest path when next sync. If any "<copyfile/>" or "<linkfile/>" were removed, the dest path will be removed in sourcecode at the same time. Bug: https://crbug.com/gerrit/11008 Change-Id: I6b7b41e94df0f9e6e52801ec755951a4c572d05d Signed-off-by: jiajia tang <tangjiajia@xiaomi.com> Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/304202 Reviewed-by: Mike Frysinger <vapier@google.com>
2021-04-25 12:02:02 +00:00
* `copy-link-files.json`: Tracking file used by `repo sync` to determine when
copyfile or linkfile are added or removed and need corresponding updates.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
* `project.list`: Tracking file used by `repo sync` to determine when projects
are added or removed and need corresponding updates in the checkout.
* `projects/`: Bare checkouts of every project synced by the manifest. The
filesystem layout matches the `<project path=...` setting in the manifest
(i.e. where it's checked out in the repo client source tree). Those
checkouts will symlink their `.git/` state to paths under here.
Some git state is further split out under `project-objects/`.
* `project-objects/`: Git objects that are safe to share across multiple
git checkouts. The filesystem layout matches the `<project name=...`
docs: improve project-objects & worktrees layout info Make it clear that the paths have a .git suffix, and clarify the reason for not using remote servers in the layout. Change-Id: I62c6977ee6f4e1e9882d45727eb239cf5489d2b6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256033 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2020-02-19 20:47:46 +00:00
setting in the manifest (i.e. the path on the remote server) with a `.git`
suffix. This allows for multiple checkouts of the same remote git repo to
share their objects. For example, you could have different branches of
switch to "main" branch for development We're migrating from "master" to "main" as the default development branch. This only affects repo itself, not manifests. Change-Id: I27489dd721c9a467a1c43736808cb3b3c1365433 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/288082 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2020-11-15 23:42:26 +00:00
`foo/bar.git` checked out to `foo/bar-main`, `foo/bar-release`, etc...
docs: improve project-objects & worktrees layout info Make it clear that the paths have a .git suffix, and clarify the reason for not using remote servers in the layout. Change-Id: I62c6977ee6f4e1e9882d45727eb239cf5489d2b6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256033 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2020-02-19 20:47:46 +00:00
There will be multiple trees under `projects/` for each one, but only one
under `project-objects/`.
This layout is designed to allow people to sync against different remotes
(e.g. a local mirror & a public review server) while avoiding duplicating
the content. However, this can run into problems if different remotes use
the same path on their respective servers. Best to avoid that.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
* `subprojects/`: Like `projects/`, but for git submodules.
* `subproject-objects/`: Like `project-objects/`, but for git submodules.
add experimental git worktree support This provides initial support for using git worktrees internally instead of our own ad-hoc symlink tree. It's been lightly tested which is why it's not currently exposed via --help. When people opt-in to worktrees in an existing repo client checkout, no projects are migrated. Instead, only new projects will use the worktree method. This allows for limited testing/opting in without having to completely blow things away or get a second checkout. Bug: https://crbug.com/gerrit/11486 Change-Id: Ic3ff891b30940a6ba497b406b2a387e0a8517ed8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254075 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
2020-02-09 07:28:34 +00:00
* `worktrees/`: Bare checkouts of every project synced by the manifest. The
filesystem layout matches the `<project name=...` setting in the manifest
docs: improve project-objects & worktrees layout info Make it clear that the paths have a .git suffix, and clarify the reason for not using remote servers in the layout. Change-Id: I62c6977ee6f4e1e9882d45727eb239cf5489d2b6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256033 Reviewed-by: Jonathan Nieder <jrn@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2020-02-19 20:47:46 +00:00
(i.e. the path on the remote server) with a `.git` suffix. This has the
same advantages as the `project-objects/` layout above.
add experimental git worktree support This provides initial support for using git worktrees internally instead of our own ad-hoc symlink tree. It's been lightly tested which is why it's not currently exposed via --help. When people opt-in to worktrees in an existing repo client checkout, no projects are migrated. Instead, only new projects will use the worktree method. This allows for limited testing/opting in without having to completely blow things away or get a second checkout. Bug: https://crbug.com/gerrit/11486 Change-Id: Ic3ff891b30940a6ba497b406b2a387e0a8517ed8 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254075 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
2020-02-09 07:28:34 +00:00
docs: add warnings about repos data model For people coming across these docs and thinking that repo's methods are good to replicate, add a note warning them against doing so. Change-Id: I443a783794313851a6e7ba1c39baebac988bff9a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298164 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2021-02-25 08:13:31 +00:00
This is used when [git worktree]'s are enabled.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
docs: add per-project review/remote/branch settings Change-Id: Iae7dc438b4a145140b4e105a61024a11e30b2c2b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255792 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-19 00:01:45 +00:00
### Global settings
docs: document internal manifests.git/config settings Change-Id: I6b32d925756375a9335522ff33376cb5f7ed1157 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254073 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-09 07:27:54 +00:00
The `.repo/manifests.git/config` file is used to track settings for the entire
repo client checkout.
sync: Added logging of repo sync state and config options for analysis. git_config.py: + Added SyncAnalysisState class, which saves the following data into the config object. ++ sys.argv, options, superproject's logging data. ++ repo.*, branch.* and remote.* parameters from config object. ++ current time as synctime. ++ Version number of the object. + All the keys for the above data are prepended with 'repo.syncstate.' + Added GetSyncAnalysisStateData and UpdateSyncAnalysisState methods to GitConfig object to save/get the above data. git_trace2_event_log.py: + Added LogConfigEvents method with code from DefParamRepoEvents to log events. sync.py: + superproject_logging_data is a dictionary that collects all the superproject data that is to be logged as trace2 event. + Sync at the end logs the previously saved syncstate.* parameters as previous_sync_state. Then it calls config's UpdateSyncAnalysisState to save and log all the current options, superproject logged data. docs/internal-fs-layout.md: + Added doc string explaining [repo.syncstate ...] sections of .repo/manifests.git/config file. test_git_config.py: + Added unit test for the new methods of GitConfig object. Tested: $ ./run_tests $ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest Tested it by running the following command multiple times. $ repo_dev sync -j 20 repo sync has finished successfully Verified config file has [syncstate ...] data saved. Bug: [google internal] b/188573450 Change-Id: I1f914ce50f3382111b72940ca56de7c41b53d460 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/313123 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Xin Li <delphij@google.com>
2021-07-28 21:36:49 +00:00
docs: document internal manifests.git/config settings Change-Id: I6b32d925756375a9335522ff33376cb5f7ed1157 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254073 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-09 07:27:54 +00:00
Most settings use the `[repo]` section to avoid conflicts with git.
sync: Added logging of repo sync state and config options for analysis. git_config.py: + Added SyncAnalysisState class, which saves the following data into the config object. ++ sys.argv, options, superproject's logging data. ++ repo.*, branch.* and remote.* parameters from config object. ++ current time as synctime. ++ Version number of the object. + All the keys for the above data are prepended with 'repo.syncstate.' + Added GetSyncAnalysisStateData and UpdateSyncAnalysisState methods to GitConfig object to save/get the above data. git_trace2_event_log.py: + Added LogConfigEvents method with code from DefParamRepoEvents to log events. sync.py: + superproject_logging_data is a dictionary that collects all the superproject data that is to be logged as trace2 event. + Sync at the end logs the previously saved syncstate.* parameters as previous_sync_state. Then it calls config's UpdateSyncAnalysisState to save and log all the current options, superproject logged data. docs/internal-fs-layout.md: + Added doc string explaining [repo.syncstate ...] sections of .repo/manifests.git/config file. test_git_config.py: + Added unit test for the new methods of GitConfig object. Tested: $ ./run_tests $ repo_dev init --use-superproject -u https://android.googlesource.com/platform/manifest Tested it by running the following command multiple times. $ repo_dev sync -j 20 repo sync has finished successfully Verified config file has [syncstate ...] data saved. Bug: [google internal] b/188573450 Change-Id: I1f914ce50f3382111b72940ca56de7c41b53d460 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/313123 Tested-by: Raman Tenneti <rtenneti@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Reviewed-by: Xin Li <delphij@google.com>
2021-07-28 21:36:49 +00:00
Everything under `[repo.syncstate.*]` is used to keep track of sync details for logging
purposes.
docs: document internal manifests.git/config settings Change-Id: I6b32d925756375a9335522ff33376cb5f7ed1157 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254073 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-09 07:27:54 +00:00
User controlled settings are initialized when running `repo init`.
init: Added --partial-clone-exclude option. partial-clone-exclude option excludes projects during partial clone. This is a comma-delimited project names (from manifest.xml). This option is persisted and it is used by the sync command. A project that has been unparital'ed will remain unpartial if that project's name is specified in the --partial-clone-exclude option. The project name should match exactly. Added $ ./run_tests -v Bug: [google internal] b/175712967 "I can't "unpartial" my androidx-main checkout" $ rm -rf androidx-main/ $ mkdir androidx-main/ $ cd androidx-main/ $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M -m default.xml $ repo_dev sync -c -j8 + Verify a project is partial $ cd frameworks/support/ $ git config -l | grep 'partial' + Unpartial a project. $ /google/bin/releases/android/git_repack/git_unpartial + Verify project is unpartial $ git config -l | grep 'partial' $ cd ../.. + Exclude the project from being unparial'ed after init and sync. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude="platform/frameworks/support,platform/frameworks/support-golden" -m default.xml + Verify project is unpartial $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. + Remove the project from exclude list and verify that project is partially cloned. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude= -m default.xml $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' Change-Id: Id5dba418eba1d3f54b54e826000406534c0ec196 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303162 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
2021-04-13 03:57:25 +00:00
| Setting | `repo init` Option | Use/Meaning |
|------------------- |---------------------------|-------------|
| manifest.groups | `--groups` & `--platform` | The manifest groups to sync |
repo: Add support for standalone manifests Added --standalone_manifest to repo tool. If set, the manifest is downloaded directly from the appropriate source (currently, we only support GS) and used instead of creating a manifest git checkout. The manifests.git repo is still created to keep track of various config but is marked as being for a standalone manifest so that the repo tool doesn't try to run networked git commands in it. BUG=b:192664812 TEST=existing tests (no coverage), manual runs Change-Id: I84378cbc7f8e515eabeccdde9665efc8cd2a9d21 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/312942 Tested-by: Jack Neus <jackneus@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
2021-07-26 23:08:54 +00:00
| manifest.standalone | `--standalone-manifest` | Download manifest as static file instead of creating checkout |
init: Added --partial-clone-exclude option. partial-clone-exclude option excludes projects during partial clone. This is a comma-delimited project names (from manifest.xml). This option is persisted and it is used by the sync command. A project that has been unparital'ed will remain unpartial if that project's name is specified in the --partial-clone-exclude option. The project name should match exactly. Added $ ./run_tests -v Bug: [google internal] b/175712967 "I can't "unpartial" my androidx-main checkout" $ rm -rf androidx-main/ $ mkdir androidx-main/ $ cd androidx-main/ $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M -m default.xml $ repo_dev sync -c -j8 + Verify a project is partial $ cd frameworks/support/ $ git config -l | grep 'partial' + Unpartial a project. $ /google/bin/releases/android/git_repack/git_unpartial + Verify project is unpartial $ git config -l | grep 'partial' $ cd ../.. + Exclude the project from being unparial'ed after init and sync. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude="platform/frameworks/support,platform/frameworks/support-golden" -m default.xml + Verify project is unpartial $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. + Remove the project from exclude list and verify that project is partially cloned. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude= -m default.xml $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' Change-Id: Id5dba418eba1d3f54b54e826000406534c0ec196 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303162 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
2021-04-13 03:57:25 +00:00
| repo.archive | `--archive` | Use `git archive` for checkouts |
| repo.clonebundle | `--clone-bundle` | Whether the initial sync used clone.bundle explicitly |
| repo.clonefilter | `--clone-filter` | Filter setting when using [partial git clones] |
| repo.depth | `--depth` | Create shallow checkouts when cloning |
| repo.dissociate | `--dissociate` | Dissociate from any reference/mirrors after initial clone |
init: add an option --enable-git-lfs-filter It was reported that git-lfs did not work with git-repo. Specifically, `git read-tree -u` run by `repo sync` would fail git-lfs's smudge filter. See https://github.com/github/git-lfs/issues/1422. In fact, by the time `git read-tree -u` is run, the repository is not bare. It is just that, the working directory is not the same as the .git directory. git-lfs's filter should work. No one seems to have delved into that issue. Today, with newer versions of git-repo and git-lfs, that issue will not reproduce. Tested with - git 2.33, git-lfs 2.13 on macOS - git 2.17, git-lfs 2.3 on ubuntu So, it seems fine to add an option --enable-git-lfs-filter, default to false, and stat that it may not work with older versions of git and git-lfs in the help doc. Bug: https://crbug.com/gerrit/14516 Change-Id: I8d21854eeeea541e072f63d6b10ad1253b1a9826 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328359 Tested-by: XD Trol <milestonejxd@gmail.com> Reviewed-by: Mike Frysinger <vapier@google.com>
2022-01-17 15:29:04 +00:00
| repo.git-lfs | `--git-lfs` | Enable [Git LFS] support |
init: Added --partial-clone-exclude option. partial-clone-exclude option excludes projects during partial clone. This is a comma-delimited project names (from manifest.xml). This option is persisted and it is used by the sync command. A project that has been unparital'ed will remain unpartial if that project's name is specified in the --partial-clone-exclude option. The project name should match exactly. Added $ ./run_tests -v Bug: [google internal] b/175712967 "I can't "unpartial" my androidx-main checkout" $ rm -rf androidx-main/ $ mkdir androidx-main/ $ cd androidx-main/ $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M -m default.xml $ repo_dev sync -c -j8 + Verify a project is partial $ cd frameworks/support/ $ git config -l | grep 'partial' + Unpartial a project. $ /google/bin/releases/android/git_repack/git_unpartial + Verify project is unpartial $ git config -l | grep 'partial' $ cd ../.. + Exclude the project from being unparial'ed after init and sync. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude="platform/frameworks/support,platform/frameworks/support-golden" -m default.xml + Verify project is unpartial $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' $ cd ../.. + Remove the project from exclude list and verify that project is partially cloned. $ repo_dev init -u https://android.googlesource.com/platform/manifest -b androidx-main --partial-clone --clone-filter=blob:limit=10M --partial-clone-exclude= -m default.xml $ repo_dev sync -c -j8 $ cd frameworks/support/ $ git config -l | grep 'partial' Change-Id: Id5dba418eba1d3f54b54e826000406534c0ec196 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/303162 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
2021-04-13 03:57:25 +00:00
| repo.mirror | `--mirror` | Checkout is a repo mirror |
| repo.partialclone | `--partial-clone` | Create [partial git clones] |
| repo.partialcloneexclude | `--partial-clone-exclude` | Comma-delimited list of project names (not paths) to exclude while using [partial git clones] |
| repo.reference | `--reference` | Reference repo client checkout |
| repo.submodules | `--submodules` | Sync git submodules |
| repo.superproject | `--use-superproject` | Sync [superproject] |
| repo.worktree | `--worktree` | Use [git worktree] for checkouts |
| user.email | `--config-name` | User's e-mail address; Copied into `.git/config` when checking out a new project |
| user.name | `--config-name` | User's name; Copied into `.git/config` when checking out a new project |
docs: document internal manifests.git/config settings Change-Id: I6b32d925756375a9335522ff33376cb5f7ed1157 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254073 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-09 07:27:54 +00:00
[partial git clones]: https://git-scm.com/docs/gitrepository-layout#_code_partialclone_code
init: added --use-superproject option to clone superproject. Added --no-use-superproject to repo and init.py to disable use of manifest superprojects. Replaced the term "sha" with "commit id". Added _GetBranch method to Superproject object. Moved shared code between init and sync into SyncSuperproject function. This function either does git clone or git fetch. If git fetch fails it does git clone. Changed Superproject constructor to accept manifest, repodir and branch to avoid passing them to multiple functions as argument. Changed functions that were raising exceptions to return either True or False. Saved the --use-superproject option in config as repo.superproject. Updated internal-fs-layout.md document. Updated the tests to work with the new API changes in Superproject. Performance for the first time sync has improved from 20 minutes to around 15 minutes. Tested the code with the following commands. $ ./run_tests -v Tested the sync code by using repo_dev alias and pointing to this CL. $ repo init took around 20 seconds longer because of cloning of superproject. $ time repo_dev init -u sso://android.git.corp.google.com/platform/manifest -b master --partial-clone --clone-filter=blob:limit=10M --repo-rev=main --use-superproject ... real 0m35.919s user 0m21.947s sys 0m8.977s First run $ time repo sync --use-superproject ... real 16m41.982s user 100m6.916s sys 19m18.753s No difference in repo sync time after the first run. Bug: [google internal] b/179090734 Bug: https://crbug.com/gerrit/13709 Bug: https://crbug.com/gerrit/13707 Change-Id: I12df92112f46e001dfbc6f12cd633c3a15cf924b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/296382 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Raman Tenneti <rtenneti@google.com>
2021-02-09 08:26:31 +00:00
[superproject]: https://en.wikibooks.org/wiki/Git/Submodules_and_Superprojects
docs: document internal manifests.git/config settings Change-Id: I6b32d925756375a9335522ff33376cb5f7ed1157 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254073 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-09 07:27:54 +00:00
docs: add per-project review/remote/branch settings Change-Id: Iae7dc438b4a145140b4e105a61024a11e30b2c2b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255792 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-19 00:01:45 +00:00
### Repo hooks settings
For more details on this feature, see the [repo-hooks docs](./repo-hooks.md).
We'll just discuss the internal configuration settings.
These are stored in the registered `<repo-hooks>` project itself, so if the
manifest switches to a different project, the settings will not be copied.
| Setting | Use/Meaning |
|--------------------------------------|-------------|
| repo.hooks.\<hook\>.approvedmanifest | User approval for secure manifest sources (e.g. https://) |
| repo.hooks.\<hook\>.approvedhash | User approval for insecure manifest sources (e.g. http://) |
For example, if our manifest had the following entries, we would store settings
under `.repo/projects/src/repohooks.git/config` (which would be reachable via
`git --git-dir=src/repohooks/.git config`).
```xml
<project path="src/repohooks" name="chromiumos/repohooks" ... />
<repo-hooks in-project="chromiumos/repohooks" ... />
```
If `<hook>` is `pre-upload`, the `.git/config` setting might be:
```ini
[repo "hooks.pre-upload"]
approvedmanifest = https://chromium.googlesource.com/chromiumos/manifest
```
## Per-project settings
These settings are somewhat meant to be tweaked by the user on a per-project
basis (e.g. `git config` in a checked out source repo).
Where possible, we re-use standard git settings to avoid confusion, and we
refrain from documenting those, so see [git-config] documentation instead.
See `repo help upload` for documentation on `[review]` settings.
The `[remote]` settings are automatically populated/updated from the manifest.
The `[branch]` settings are updated by `repo start` and `git branch`.
upload: Allow user to configure unusual commit threshold Add a per-remote option `uploadwarningthreshold` allowing the user to override how many commits can be uploaded prior to a warning being displayed. Change-Id: Ia7e1b2c7de89a0bf9ca1c24cc83dc595b3667437 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/354375 Tested-by: David Greenaway <dgreenaway@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
2022-12-08 22:38:24 +00:00
| Setting | Subcommands | Use/Meaning |
|---------------------------------------|---------------|-------------|
| review.\<url\>.autocopy | upload | Automatically add to `--cc=<value>` |
| review.\<url\>.autoreviewer | upload | Automatically add to `--reviewers=<value>` |
| review.\<url\>.autoupload | upload | Automatically answer "yes" or "no" to all prompts |
| review.\<url\>.uploadhashtags | upload | Automatically add to `--hashtag=<value>` |
| review.\<url\>.uploadlabels | upload | Automatically add to `--label=<value>` |
| review.\<url\>.uploadnotify | upload | [Notify setting][upload-notify] to use |
| review.\<url\>.uploadtopic | upload | Default [topic] to use |
| review.\<url\>.uploadwarningthreshold | upload | Warn when attempting to upload more than this many CLs |
| review.\<url\>.username | upload | Override username with `ssh://` review URIs |
| remote.\<remote\>.fetch | sync | Set of refs to fetch |
| remote.\<remote\>.projectname | \<network\> | The name of the project as it exists in Gerrit review |
| remote.\<remote\>.pushurl | upload | The base URI for pushing CLs |
| remote.\<remote\>.review | upload | The URI of the Gerrit review server |
| remote.\<remote\>.url | sync & upload | The URI of the git project to fetch |
| branch.\<branch\>.merge | sync & upload | The branch to merge & upload & track |
| branch.\<branch\>.remote | sync & upload | The remote to track |
docs: add per-project review/remote/branch settings Change-Id: Iae7dc438b4a145140b4e105a61024a11e30b2c2b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255792 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-19 00:01:45 +00:00
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
## ~/ dotconfig layout
Enable use of REPO_CONFIG_DIR to customize .repoconfig location For use cases with multiple instances of repo, eg some CI environments. Bug: https://crbug.com/gerrit/15803 Change-Id: I65c1cfc8f6a98adfeb5efefc7ac6b45bf8e134de Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/356719 Tested-by: Gavin Mak <gavinmak@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
2023-01-26 23:27:51 +00:00
Repo will create & maintain a few files under the `.repoconfig/` directory.
This is placed in the user's home directory by default but can be changed by
setting `REPO_CONFIG_DIR`.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
* `.repoconfig/`: Repo's per-user directory for all random config files/state.
git_config: add support for repo-specific settings This allows people to write ~/.repoconfig/config akin to ~/.gitconfig and .repo/config akin to .git/config. This allows us to add settings specific to repo without mixing up git, and to persist in general. Change-Id: I1c6fbe31e63fb8ce26aa85335349c6ae5b1712c6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255832 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
2020-02-19 02:31:51 +00:00
* `.repoconfig/config`: Per-user settings using [git-config] file format.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
* `.repoconfig/keyring-version`: Cache file for checking if the gnupg subdir
has all the same keys as the repo launcher. Used to avoid running gpg
constantly as that can be quite slow.
* `.repoconfig/gnupg/`: GnuPG's internal state directory used when repo needs
to run `gpg`. This provides isolation from the user's normal `~/.gnupg/`.
git_config: add support for repo-specific settings This allows people to write ~/.repoconfig/config akin to ~/.gitconfig and .repo/config akin to .git/config. This allows us to add settings specific to repo without mixing up git, and to persist in general. Change-Id: I1c6fbe31e63fb8ce26aa85335349c6ae5b1712c6 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255832 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
2020-02-19 02:31:51 +00:00
* `.repoconfig/.repo_config.json`: JSON cache of the `.repoconfig/config`
file for repo to read/process quickly.
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
* `.repo_.gitconfig.json`: JSON cache of the `.gitconfig` file for repo to
read/process quickly.
docs: add per-project review/remote/branch settings Change-Id: Iae7dc438b4a145140b4e105a61024a11e30b2c2b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255792 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-19 00:01:45 +00:00
[git-config]: https://git-scm.com/docs/git-config
init: add an option --enable-git-lfs-filter It was reported that git-lfs did not work with git-repo. Specifically, `git read-tree -u` run by `repo sync` would fail git-lfs's smudge filter. See https://github.com/github/git-lfs/issues/1422. In fact, by the time `git read-tree -u` is run, the repository is not bare. It is just that, the working directory is not the same as the .git directory. git-lfs's filter should work. No one seems to have delved into that issue. Today, with newer versions of git-repo and git-lfs, that issue will not reproduce. Tested with - git 2.33, git-lfs 2.13 on macOS - git 2.17, git-lfs 2.3 on ubuntu So, it seems fine to add an option --enable-git-lfs-filter, default to false, and stat that it may not work with older versions of git and git-lfs in the help doc. Bug: https://crbug.com/gerrit/14516 Change-Id: I8d21854eeeea541e072f63d6b10ad1253b1a9826 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328359 Tested-by: XD Trol <milestonejxd@gmail.com> Reviewed-by: Mike Frysinger <vapier@google.com>
2022-01-17 15:29:04 +00:00
[Git LFS]: https://git-lfs.github.com/
docs: add warnings about repos data model For people coming across these docs and thinking that repo's methods are good to replicate, add a note warning them against doing so. Change-Id: I443a783794313851a6e7ba1c39baebac988bff9a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298164 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2021-02-25 08:13:31 +00:00
[git worktree]: https://git-scm.com/docs/git-worktree
[gitsubmodules]: https://git-scm.com/docs/gitsubmodules
docs: add a developer reference for .repo/ paths Currently the only reference for these is the source which can be a pita when needing to refer to something quickly. Change-Id: I52baeb9a4935814cf99fa9a9b3102e8e46cddb0d Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253972 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-08 00:44:53 +00:00
[manifest-format.md]: ./manifest-format.md
[local manifests]: ./manifest-format.md#Local-Manifests
docs: add warnings about repos data model For people coming across these docs and thinking that repo's methods are good to replicate, add a note warning them against doing so. Change-Id: I443a783794313851a6e7ba1c39baebac988bff9a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/298164 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
2021-02-25 08:13:31 +00:00
[superprojects]: https://en.wikibooks.org/wiki/Git/Submodules_and_Superprojects
docs: add per-project review/remote/branch settings Change-Id: Iae7dc438b4a145140b4e105a61024a11e30b2c2b Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255792 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-19 00:01:45 +00:00
[topic]: https://gerrit-review.googlesource.com/Documentation/intro-user.html#topics
upload: add config setting for upload notifications This allows people to set default e-mail notifications via gitconfig. Bug: https://crbug.com/gerrit/12451 Change-Id: Ic04ea3b7df0c5603c491961112c5be8cabb9dddd Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/259014 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-03-14 21:39:03 +00:00
[upload-notify]: https://gerrit-review.googlesource.com/Documentation/user-upload.html#notify
Reference in New Issue Copy Permalink