Compare commits

..

17 Commits
v2.41 ... v2.45

Author SHA1 Message Date
73356f1d5c project: Check if dotgit exists w/out symlink check
os.path.exists returns false on a broken symlink. This is not what repo
needs when checking if a project is setup properly.

For example, if src/foo/.git can't be resolved, repo tries to create
symlink and that results in FileExistsError.

Use lexists which returns True even if symlink is broken. That will
force path where repo checks where symlink is pointing to and will fix
it to the correct location.

Bug: b/281746795
Change-Id: Id3f7dc3a3cb6499d02ce7335eca992ddc7deb645
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415197
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: George Engelbrecht <engeg@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
2024-03-20 22:09:14 +00:00
09fc214a79 git: raise soft version to 2.7.4
git-1.9.1 was released in 2014 while git-2.7.4 was released in 2016.
Debian Stretch was released in 2017 and Ubuntu Xenial was released in
2016 which are plenty old at this point.  Both of those include at
least git-2.7.4.

We will start warning users of Debian Jessie (released in 2015 & EOL
in 2020) and Ubuntu Trusty (released in 2014 & EOL Apr 2024) that
they will need to upgrade.

Change-Id: I6be3809bc45968fdcb02cad3f7daf75ded1bb5b1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415137
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-03-20 21:11:26 +00:00
3762b17e98 git: raise hard version to 1.7.9
Debian 6 Squeeze went EOL in Feb 2016.  We don't need to carry support
for that anymore as there have been 6 major releases since.  That means
we can bump the required git version from 1.7.2 to 1.7.9.  Ubuntu Precise
shipped with the latter.

git-1.7.2 was released in 2010 while git-1.7.9 was released in 2012.
So that shouldn't be a problem either.  And we've been warning people
using git versions older than 1.9.1 for 3 years now that they need to
upgrade.

Change-Id: I7712f110ea158297b489b8379b112c6700b21a46
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415097
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-03-20 19:49:44 +00:00
ae419e1e01 docs: release: add recent git/python/ssh/debian info
Change-Id: I744360b1bfc816e94b3511f0130abb2c08dedd42
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415117
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-03-20 19:49:01 +00:00
a3a7372612 main: Stringify project name in error_info
If a project can't be removed from checkout due to uncommitted changes
present, error.project is type of Project and not a string (as it is in
some cases). Project is not JSON serializable, resulting in exception
within exception handler:

TypeError: Object of type Project is not JSON serializable

This change casts project to string as a defensive mechanism. It also
passes project name instead of project object.

Change-Id: Ie7b782d73dc3647975755d5a3774d16ea6cd5348
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/413877
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-03-15 19:26:10 +00:00
fff1d2d74c ssh: Print details if running the command fails
Change-Id: I87adbdd5fe4eb2709c97ab4c21b414145acf788b
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/392915
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Tuan Vo Hung <vohungtuan@gmail.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-03-11 16:40:55 +00:00
4b01a242d8 upload: Fix patchset description destination
Bug: 308467447
Change-Id: I8ad598d39f5fdb24d549d3277ad5fedac203581b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/412477
Reviewed-by: George Engelbrecht <engeg@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-03-08 18:05:36 +00:00
46790229fc sync: Fix sorting for nested projects
The current logic to create checkout layers doesn't work in all cases.
For example, let's assume there are three projects: "foo", "foo/bar" and
"foo-bar". Sorting lexicographical order is incorrect as foo-bar would
be placed between foo and foo/bar, breaking layering logic.

Instead, we split filepaths based using path delimiter (always /) and
then use lexicographical sort.

BUG=b:325119758
TEST=./run_tests, manual sync on chromiumos repository

Change-Id: I76924c3cc6ba2bb860d7a3e48406a6bba8f58c10
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/412338
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: George Engelbrecht <engeg@google.com>
2024-03-08 17:58:24 +00:00
edadb25c02 sync: introduce --force-checkout
In some cases (e.g. in a CI system), it's desirable to be able to
instruct repo to force checkout. This flag passes --force flag to `git
checkout` operations.

Bug: b/327624021
Change-Id: I579edda546fb8147c4e1a267e2605fcf6e597421
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/411518
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: George Engelbrecht <engeg@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
2024-03-07 17:21:51 +00:00
96edb9b573 upload: Add support for setting patchset description
Bug: 308467447
Change-Id: I7abcbc98131b826120fc9ab85d5b889f90db4b0a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/355968
Tested-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
2024-03-04 18:50:24 +00:00
5554572f02 sync: Introduce git checkout levels
If a repo manifest is updated so that project B is placed within a
project A, and if project A had content in new B's location in the old
checkout, then repo sync could break depending on checkout order, since
B can't be checked out before A.

This change introduces checkout levels which enforces right sequence of
checkouts while still allowing for parallel checkout. In an example
above, A will always be checked out first before B.

BUG=b:325119758
TEST=./run_tests, manual sync on ChromeOS repository

Change-Id: Ib3b5e4d2639ca56620a1e4c6bf76d7b1ab805250
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410421
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2024-02-27 17:28:33 +00:00
97ca50f5f9 git_command: Return None from GetEventTargetPath() if set to empty string
If trace2.eventTarget was set to the empty string,
match git behavior and don't write a trace.

Bug: 319673783
Change-Id: I02b3884ad97551f8a9d7363c2cbe6b0adee6f73e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410518
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Peter Collingbourne <pcc@google.com>
2024-02-26 17:51:11 +00:00
8896b68926 trace: Save trace2 sid in REPO_TRACE file
git-trace2 events contain additional information what git is doing under
the hood, and repo doesn't have visibility into.

Instead of relying on timestamp information to match REPO_TRACE with
git-trace2 events, add SID information into REPO_TRACE.

Change-Id: I37672a3face81858072c7a3ce34ca3379199dab5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410280
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-02-22 20:55:09 +00:00
fec8cd6704 subcmds: sync: Remove deprecated _AUTO_GC
Opportunistic cleanup. It looks like this deprecated feature was slated
for deletion nearly a year ago.

Bug: None
Test: ./run_tests
Change-Id: I0bd0c0e6acbd1eaee1c0b4945c79038257d22f44
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410198
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Tested-by: Greg Edelston <gredelston@google.com>
2024-02-20 19:55:15 +00:00
b8139bdcf8 launcher: Set shebang to python3
Some (most?) Linux repos don't have /usr/bin/python, unless
python-is-python3 is installed. While package owners can adjust shebang,
we have seen an increase in number of bugs filed as extra steps are
required.

Per PEP-0394, python3 is acceptable and should be available if python3
is supported. We no longer support python2, and repo no longer works
with python2, so this change makes that explicit.

Bug: 40014585
Change-Id: I9aed90fd470ef601bd33bd596af3df69da69ef5d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/407497
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Максим Паймушкин <maxim.paymushkin@gmail.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
2024-02-07 20:44:32 +00:00
26fa3180fb sync: ensure RepoChangedException propagated
Prior to this change RepoChangedException would be caught and re-rasied
as a different exception. This would prevent RepoChangedException
handler from running in main.py

Bug: b/323232806
Change-Id: I9055ff95d439d6ff225206c5bf1755cc718bcfcc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/407144
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-02-06 18:46:19 +00:00
d379e77f44 stop passing project to UpdateManifestError
UpdateManifestError inherits from RepoExitError which inherits
from BaseRepoError. None of them takes project as kwargs
causing the error like "UpdateManifestError() takes no keyword
arguments" in b/317183321

[1]: https://gerrit.googlesource.com/git-repo/+/449b23b698d7d4b13909667a49a0698eb495eeaa/error.py#144

Bug: b/317183321
Change-Id: I64c3dc502027f9dda56a0824f2712364b4502934
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/398997
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
Tested-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Jason Chang <jasonnc@google.com>
2024-02-02 18:35:13 +00:00
11 changed files with 250 additions and 60 deletions

View File

@ -202,7 +202,7 @@ still support them.
Things in italics are things we used to care about but probably don't anymore.
| Date | EOL | [Git][rel-g] | [Python][rel-p] | [SSH][rel-o] | [Ubuntu][rel-u] / [Debian][rel-d] | Git | Python | SSH |
|:--------:|:------------:|:------------:|:---------------:|:------------:|-----------------------------------|-----|--------|-----|
|:--------:|:------------:|:------------:|:---------------:|:------------:|-----------------------------------|:---:|:------:|:---:|
| Apr 2008 | | | | 5.0 |
| Jun 2008 | | | | 5.1 |
| Oct 2008 | *Oct 2013* | | 2.6.0 | | *10.04 Lucid* - 10.10 Maverick / *Squeeze* |
@ -241,7 +241,7 @@ Things in italics are things we used to care about but probably don't anymore.
| Feb 2014 | *Dec 2014* | **1.9.0** | | | *14.04 Trusty* |
| Mar 2014 | *Mar 2019* | | *3.4.0* | | *14.04 Trusty* - 15.10 Wily / *Jessie* |
| Mar 2014 | | | | 6.6 | *14.04 Trusty* - 14.10 Utopic |
| Apr 2014 | *Apr 2022* | | | | *14.04 Trusty* | 1.9.1 | 2.7.5 3.4.0 | 6.6 |
| Apr 2014 | *Apr 2024* | | | | *14.04 Trusty* | 1.9.1 | 2.7.5 3.4.0 | 6.6 |
| May 2014 | *Dec 2014* | 2.0.0 |
| Aug 2014 | *Dec 2014* | *2.1.0* | | | 14.10 Utopic - 15.04 Vivid / *Jessie* |
| Oct 2014 | | | | 6.7 | 15.04 Vivid |
@ -262,7 +262,7 @@ Things in italics are things we used to care about but probably don't anymore.
| Jan 2016 | *Jul 2017* | *2.7.0* | | | *16.04 Xenial* |
| Feb 2016 | | | | 7.2 | *16.04 Xenial* |
| Mar 2016 | *Jul 2017* | 2.8.0 |
| Apr 2016 | *Apr 2024* | | | | *16.04 Xenial* | 2.7.4 | 2.7.11 3.5.1 | 7.2 |
| Apr 2016 | *Apr 2026* | | | | *16.04 Xenial* | 2.7.4 | 2.7.11 3.5.1 | 7.2 |
| Jun 2016 | *Jul 2017* | 2.9.0 | | | 16.10 Yakkety |
| Jul 2016 | | | | 7.3 | 16.10 Yakkety |
| Sep 2016 | *Sep 2017* | 2.10.0 |
@ -312,14 +312,33 @@ Things in italics are things we used to care about but probably don't anymore.
| Oct 2020 | | | | | 20.10 Groovy | 2.27.0 | 2.7.18 3.8.6 | 8.3 |
| Oct 2020 | **Oct 2025** | | 3.9.0 | | 21.04 Hirsute / **Bullseye** |
| Dec 2020 | *Mar 2021* | 2.30.0 | | | 21.04 Hirsute / **Bullseye** |
| Mar 2021 | | 2.31.0 |
| Mar 2021 | | | | 8.5 |
| Mar 2021 | | 2.31.0 | | 8.5 |
| Apr 2021 | | | | 8.6 |
| Apr 2021 | *Jan 2022* | | | | 21.04 Hirsute | 2.30.2 | 2.7.18 3.9.4 | 8.4 |
| Jun 2021 | | 2.32.0 |
| Aug 2021 | | 2.33.0 |
| Aug 2021 | | | | 8.7 |
| Aug 2021 | | 2.33.0 | | 8.7 |
| Aug 2021 | **Aug 2026** | | | | **Debian 11 Bullseye** | 2.30.2 | 2.7.18 3.9.2 | 8.4 |
| Sep 2021 | | | | 8.8 |
| Oct 2021 | | 2.34.0 | 3.10.0 | | **22.04 Jammy** |
| Jan 2022 | | 2.35.0 |
| Feb 2022 | | | | 8.9 | **22.04 Jammy** |
| Apr 2022 | | 2.36.0 | | 9.0 |
| Apr 2022 | **Apr 2032** | | | | **22.04 Jammy** | 2.34.1 | 2.7.18 3.10.6 | 8.9 |
| Jun 2022 | | 2.37.0 |
| Oct 2022 | | 2.38.0 | | 9.1 |
| Oct 2022 | | | 3.11.0 | | **Bookworm** |
| Dec 2022 | | 2.39.0 | | | **Bookworm** |
| Feb 2023 | | | | 9.2 | **Bookworm** |
| Mar 2023 | | 2.40.0 | | 9.3 |
| Jun 2023 | | 2.41.0 |
| Jun 2023 | **Jun 2028** | | | | **Debian 12 Bookworm** | 2.39.2 | 3.11.2 | 9.2 |
| Aug 2023 | | 2.42.0 | | 9.4 |
| Oct 2023 | | | 3.12.0 | 9.5 |
| Nov 2022 | | 2.43.0 |
| Dec 2023 | | | | 9.6 |
| Feb 2024 | | 2.44.0 |
| Mar 2024 | | | | 9.7 |
| Oct 2024 | | | 3.13.0 |
| **Date** | **EOL** | **[Git][rel-g]** | **[Python][rel-p]** | **[SSH][rel-o]** | **[Ubuntu][rel-u] / [Debian][rel-d]** | **Git** | **Python** | **SSH** |
@ -328,7 +347,7 @@ Things in italics are things we used to care about but probably don't anymore.
[rel-g]: https://en.wikipedia.org/wiki/Git#Releases
[rel-o]: https://www.openssh.com/releasenotes.html
[rel-p]: https://en.wikipedia.org/wiki/History_of_Python#Table_of_versions
[rel-u]: https://en.wikipedia.org/wiki/Ubuntu_version_history#Table_of_versions
[rel-u]: https://wiki.ubuntu.com/Releases
[example announcement]: https://groups.google.com/d/topic/repo-discuss/UGBNismWo1M/discussion
[repo-discuss@googlegroups.com]: https://groups.google.com/forum/#!forum/repo-discuss
[go/repo-release]: https://goto.google.com/repo-release

View File

@ -135,6 +135,8 @@ def GetEventTargetPath():
if retval == 0:
# Strip trailing carriage-return in path.
path = p.stdout.rstrip("\n")
if path == "":
return None
elif retval != 1:
# `git config --get` is documented to produce an exit status of `1`
# if the requested variable is not present in the configuration.

13
main.py
View File

@ -270,10 +270,14 @@ class _Repo:
self._PrintHelp(short=True)
return 1
run = lambda: self._RunLong(name, gopts, argv) or 0
git_trace2_event_log = EventLog()
run = (
lambda: self._RunLong(name, gopts, argv, git_trace2_event_log) or 0
)
with Trace(
"starting new command: %s",
"starting new command: %s [sid=%s]",
", ".join([name] + argv),
git_trace2_event_log.full_sid,
first_trace=True,
):
if gopts.trace_python:
@ -290,12 +294,11 @@ class _Repo:
result = run()
return result
def _RunLong(self, name, gopts, argv):
def _RunLong(self, name, gopts, argv, git_trace2_event_log):
"""Execute the (longer running) requested subcommand."""
result = 0
SetDefaultColoring(gopts.color)
git_trace2_event_log = EventLog()
outer_client = RepoClient(self.repodir)
repo_client = outer_client
if gopts.submanifest_path:
@ -422,7 +425,7 @@ class _Repo:
error_info = json.dumps(
{
"ErrorType": type(error).__name__,
"Project": project,
"Project": str(project),
"Message": str(error),
}
)

View File

@ -21,6 +21,7 @@ import random
import re
import shutil
import stat
import string
import subprocess
import sys
import tarfile
@ -266,6 +267,7 @@ class ReviewableBranch:
dest_branch=None,
validate_certs=True,
push_options=None,
patchset_description=None,
):
self.project.UploadForReview(
branch=self.name,
@ -281,6 +283,7 @@ class ReviewableBranch:
dest_branch=dest_branch,
validate_certs=validate_certs,
push_options=push_options,
patchset_description=patchset_description,
)
def GetPublishedRefs(self):
@ -1089,6 +1092,7 @@ class Project:
dest_branch=None,
validate_certs=True,
push_options=None,
patchset_description=None,
):
"""Uploads the named branch for code review."""
if branch is None:
@ -1171,6 +1175,10 @@ class Project:
opts += ["wip"]
if ready:
opts += ["ready"]
if patchset_description:
opts += [
f"m={self._encode_patchset_description(patchset_description)}"
]
if opts:
ref_spec = ref_spec + "%" + ",".join(opts)
cmd.append(ref_spec)
@ -1183,6 +1191,30 @@ class Project:
R_PUB + branch.name, R_HEADS + branch.name, message=msg
)
@staticmethod
def _encode_patchset_description(original):
"""Applies percent-encoding for strings sent as patchset description.
The encoding used is based on but stricter than URL encoding (Section
2.1 of RFC 3986). The only non-escaped characters are alphanumerics, and
'SPACE' (U+0020) can be represented as 'LOW LINE' (U+005F) or
'PLUS SIGN' (U+002B).
For more information, see the Gerrit docs here:
https://gerrit-review.googlesource.com/Documentation/user-upload.html#patch_set_description
"""
SAFE = {ord(x) for x in string.ascii_letters + string.digits}
def _enc(b):
if b in SAFE:
return chr(b)
elif b == ord(" "):
return "_"
else:
return f"%{b:02x}"
return "".join(_enc(x) for x in original.encode("utf-8"))
def _ExtractArchive(self, tarpath, path=None):
"""Extract the given tar on its current location
@ -1483,6 +1515,7 @@ class Project:
self,
syncbuf,
force_sync=False,
force_checkout=False,
submodules=False,
errors=None,
verbose=False,
@ -1570,7 +1603,7 @@ class Project:
syncbuf.info(self, "discarding %d commits", len(lost))
try:
self._Checkout(revid, quiet=True)
self._Checkout(revid, force_checkout=force_checkout, quiet=True)
if submodules:
self._SyncSubmodules(quiet=True)
except GitError as e:
@ -1780,11 +1813,11 @@ class Project:
)
else:
msg = (
"error: %s: Cannot remove project: uncommitted"
"error: %s: Cannot remove project: uncommitted "
"changes are present.\n" % self.RelPath(local=False)
)
logger.error(msg)
raise DeleteDirtyWorktreeError(msg, project=self)
raise DeleteDirtyWorktreeError(msg, project=self.name)
if verbose:
print(f"{self.RelPath(local=False)}: Deleting obsolete checkout.")
@ -2825,10 +2858,12 @@ class Project:
except OSError:
return False
def _Checkout(self, rev, quiet=False):
def _Checkout(self, rev, force_checkout=False, quiet=False):
cmd = ["checkout"]
if quiet:
cmd.append("-q")
if force_checkout:
cmd.append("-f")
cmd.append(rev)
cmd.append("--")
if GitCommand(self, cmd).Wait() != 0:
@ -3306,7 +3341,7 @@ class Project:
if not platform_utils.islink(dotgit) and platform_utils.isdir(dotgit):
self._MigrateOldWorkTreeGitDir(dotgit, project=self.name)
init_dotgit = not os.path.exists(dotgit)
init_dotgit = not os.path.lexists(dotgit)
if self.use_git_worktrees:
if init_dotgit:
self._InitGitWorktree()

9
repo
View File

@ -1,4 +1,4 @@
#!/usr/bin/env python
#!/usr/bin/env python3
#
# Copyright (C) 2008 The Android Open Source Project
#
@ -124,7 +124,7 @@ if not REPO_REV:
BUG_URL = "https://issues.gerritcodereview.com/issues/new?component=1370071"
# increment this whenever we make important changes to this script
VERSION = (2, 40)
VERSION = (2, 45)
# increment this if the MAINTAINER_KEYS block is modified
KEYRING_VERSION = (2, 3)
@ -210,9 +210,8 @@ GIT = "git" # our git command
# NB: The version of git that the repo launcher requires may be much older than
# the version of git that the main repo source tree requires. Keeping this at
# an older version also makes it easier for users to upgrade/rollback as needed.
#
# git-1.7 is in (EOL) Ubuntu Precise.
MIN_GIT_VERSION = (1, 7, 2) # minimum supported git version
# See requirements.json for versions.
MIN_GIT_VERSION = (1, 7, 9) # minimum supported git version
repodir = ".repo" # name of repo's private directory
S_repo = "repo" # special repo repository
S_manifests = "manifests" # special manifest repository

View File

@ -46,12 +46,16 @@
# Supported git versions.
#
# git-1.7.2 is in Debian Squeeze.
# git-1.7.9 is in Ubuntu Precise.
# git-1.9.1 is in Ubuntu Trusty.
# git-1.7.10 is in Debian Wheezy.
# git-1.9.1 is in Ubuntu Trusty.
# git-2.1.4 is in Debian Jessie.
# git-2.7.4 is in Ubuntu Xenial.
# git-2.11.0 is in Debian Stretch.
# git-2.17.0 is in Ubuntu Bionic.
# git-2.20.1 is in Debian Buster.
"git": {
"hard": [1, 7, 2],
"soft": [1, 9, 1]
"hard": [1, 7, 9],
"soft": [2, 7, 4]
}
}

8
ssh.py
View File

@ -57,8 +57,12 @@ def version():
except FileNotFoundError:
print("fatal: ssh not installed", file=sys.stderr)
sys.exit(1)
except subprocess.CalledProcessError:
print("fatal: unable to detect ssh version", file=sys.stderr)
except subprocess.CalledProcessError as e:
print(
"fatal: unable to detect ssh version"
f" (code={e.returncode}, output={e.stdout})",
file=sys.stderr,
)
sys.exit(1)

View File

@ -21,6 +21,7 @@ import multiprocessing
import netrc
import optparse
import os
from pathlib import Path
import sys
import tempfile
import time
@ -82,16 +83,54 @@ from wrapper import Wrapper
_ONE_DAY_S = 24 * 60 * 60
# Env var to implicitly turn auto-gc back on. This was added to allow a user to
# revert a change in default behavior in v2.29.9. Remove after 2023-04-01.
_REPO_AUTO_GC = "REPO_AUTO_GC"
_AUTO_GC = os.environ.get(_REPO_AUTO_GC) == "1"
_REPO_ALLOW_SHALLOW = os.environ.get("REPO_ALLOW_SHALLOW")
logger = RepoLogger(__file__)
def _SafeCheckoutOrder(checkouts: List[Project]) -> List[List[Project]]:
"""Generate a sequence of checkouts that is safe to perform. The client
should checkout everything from n-th index before moving to n+1.
This is only useful if manifest contains nested projects.
E.g. if foo, foo/bar and foo/bar/baz are project paths, then foo needs to
finish before foo/bar can proceed, and foo/bar needs to finish before
foo/bar/baz."""
res = [[]]
current = res[0]
# depth_stack contains a current stack of parent paths.
depth_stack = []
# Checkouts are iterated in the hierarchical order. That way, it can easily
# be determined if the previous checkout is parent of the current checkout.
# We are splitting by the path separator so the final result is
# hierarchical, and not just lexicographical. For example, if the projects
# are: foo, foo/bar, foo-bar, lexicographical order produces foo, foo-bar
# and foo/bar, which doesn't work.
for checkout in sorted(checkouts, key=lambda x: x.relpath.split("/")):
checkout_path = Path(checkout.relpath)
while depth_stack:
try:
checkout_path.relative_to(depth_stack[-1])
except ValueError:
# Path.relative_to returns ValueError if paths are not relative.
# TODO(sokcevic): Switch to is_relative_to once min supported
# version is py3.9.
depth_stack.pop()
else:
if len(depth_stack) >= len(res):
# Another depth created.
res.append([])
break
current = res[len(depth_stack)]
current.append(checkout)
depth_stack.append(checkout_path)
return res
class _FetchOneResult(NamedTuple):
"""_FetchOne return value.
@ -243,6 +282,11 @@ directories if they have previously been linked to a different
object directory. WARNING: This may cause data to be lost since
refs may be removed when overwriting.
The --force-checkout option can be used to force git to switch revs even if the
index or the working tree differs from HEAD, and if there are untracked files.
WARNING: This may cause data to be lost since uncommitted changes may be
removed.
The --force-remove-dirty option can be used to remove previously used
projects with uncommitted changes. WARNING: This may cause data to be
lost since uncommitted changes may be removed with projects that no longer
@ -340,6 +384,14 @@ later is required to fix a server side protocol bug.
"point to a different object directory. WARNING: this "
"may cause loss of data",
)
p.add_option(
"--force-checkout",
dest="force_checkout",
action="store_true",
help="force checkout even if it results in throwing away "
"uncommitted modifications. "
"WARNING: this may cause loss of data",
)
p.add_option(
"--force-remove-dirty",
dest="force_remove_dirty",
@ -956,12 +1008,17 @@ later is required to fix a server side protocol bug.
return _FetchMainResult(all_projects)
def _CheckoutOne(self, detach_head, force_sync, verbose, project):
def _CheckoutOne(
self, detach_head, force_sync, force_checkout, verbose, project
):
"""Checkout work tree for one project
Args:
detach_head: Whether to leave a detached HEAD.
force_sync: Force checking out of the repo.
force_sync: Force checking out of .git directory (e.g. overwrite
existing git directory that was previously linked to a different
object directory).
force_checkout: Force checking out of the repo content.
verbose: Whether to show verbose messages.
project: Project object for the project to checkout.
@ -976,7 +1033,11 @@ later is required to fix a server side protocol bug.
errors = []
try:
project.Sync_LocalHalf(
syncbuf, force_sync=force_sync, errors=errors, verbose=verbose
syncbuf,
force_sync=force_sync,
force_checkout=force_checkout,
errors=errors,
verbose=verbose,
)
success = syncbuf.Finish()
except GitError as e:
@ -1040,15 +1101,22 @@ later is required to fix a server side protocol bug.
pm.update(msg=project.name)
return ret
proc_res = self.ExecuteInParallel(
opt.jobs_checkout,
functools.partial(
self._CheckoutOne, opt.detach_head, opt.force_sync, opt.verbose
),
all_projects,
callback=_ProcessResults,
output=Progress("Checking out", len(all_projects), quiet=opt.quiet),
)
for projects in _SafeCheckoutOrder(all_projects):
proc_res = self.ExecuteInParallel(
opt.jobs_checkout,
functools.partial(
self._CheckoutOne,
opt.detach_head,
opt.force_sync,
opt.force_checkout,
opt.verbose,
),
projects,
callback=_ProcessResults,
output=Progress(
"Checking out", len(all_projects), quiet=opt.quiet
),
)
self._local_sync_state.Save()
return proc_res and not err_results
@ -1544,9 +1612,7 @@ later is required to fix a server side protocol bug.
mp, event_log.TASK_SYNC_LOCAL, start, time.time(), clean
)
if not clean:
raise UpdateManifestError(
aggregate_errors=errors, project=mp.name
)
raise UpdateManifestError(aggregate_errors=errors)
self._ReloadManifest(manifest_name, mp.manifest)
def ValidateOptions(self, opt, args):
@ -1577,16 +1643,6 @@ later is required to fix a server side protocol bug.
if opt.prune is None:
opt.prune = True
if opt.auto_gc is None and _AUTO_GC:
logger.error(
"Will run `git gc --auto` because %s is set. %s is deprecated "
"and will be removed in a future release. Use `--auto-gc` "
"instead.",
_REPO_AUTO_GC,
_REPO_AUTO_GC,
)
opt.auto_gc = True
def _ValidateOptionsWithManifest(self, opt, mp):
"""Like ValidateOptions, but after we've updated the manifest.
@ -1630,7 +1686,7 @@ later is required to fix a server side protocol bug.
errors = []
try:
self._ExecuteHelper(opt, args, errors)
except RepoExitError:
except (RepoExitError, RepoChangedException):
raise
except (KeyboardInterrupt, Exception) as e:
raise RepoUnhandledExceptionError(e, aggregate_errors=errors)

View File

@ -244,6 +244,12 @@ Gerrit Code Review: https://www.gerritcodereview.com/
default=[],
help="add a label when uploading",
)
p.add_option(
"--pd",
"--patchset-description",
dest="patchset_description",
help="description for patchset",
)
p.add_option(
"--re",
"--reviewers",
@ -655,6 +661,7 @@ Gerrit Code Review: https://www.gerritcodereview.com/
dest_branch=destination,
validate_certs=opt.validate_certs,
push_options=opt.push_options,
patchset_description=opt.patchset_description,
)
branch.uploaded = True

View File

@ -107,6 +107,16 @@ class ReviewableBranchTests(unittest.TestCase):
self.assertTrue(rb.date)
class ProjectTests(unittest.TestCase):
"""Check Project behavior."""
def test_encode_patchset_description(self):
self.assertEqual(
project.Project._encode_patchset_description("abcd00!! +"),
"abcd00%21%21_%2b",
)
class CopyLinkTestCase(unittest.TestCase):
"""TestCase for stub repo client checkouts.

View File

@ -304,6 +304,57 @@ class LocalSyncState(unittest.TestCase):
self.assertEqual(self.state.GetFetchTime(projA), 5)
class FakeProject:
def __init__(self, relpath):
self.relpath = relpath
def __str__(self):
return f"project: {self.relpath}"
def __repr__(self):
return str(self)
class SafeCheckoutOrder(unittest.TestCase):
def test_no_nested(self):
p_f = FakeProject("f")
p_foo = FakeProject("foo")
out = sync._SafeCheckoutOrder([p_f, p_foo])
self.assertEqual(out, [[p_f, p_foo]])
def test_basic_nested(self):
p_foo = p_foo = FakeProject("foo")
p_foo_bar = FakeProject("foo/bar")
out = sync._SafeCheckoutOrder([p_foo, p_foo_bar])
self.assertEqual(out, [[p_foo], [p_foo_bar]])
def test_complex_nested(self):
p_foo = FakeProject("foo")
p_foobar = FakeProject("foobar")
p_foo_dash_bar = FakeProject("foo-bar")
p_foo_bar = FakeProject("foo/bar")
p_foo_bar_baz_baq = FakeProject("foo/bar/baz/baq")
p_bar = FakeProject("bar")
out = sync._SafeCheckoutOrder(
[
p_foo_bar_baz_baq,
p_foo,
p_foobar,
p_foo_dash_bar,
p_foo_bar,
p_bar,
]
)
self.assertEqual(
out,
[
[p_bar, p_foo, p_foo_dash_bar, p_foobar],
[p_foo_bar],
[p_foo_bar_baz_baq],
],
)
class GetPreciousObjectsState(unittest.TestCase):
"""Tests for _GetPreciousObjectsState."""