Commit Graph

1900 Commits

Author SHA1 Message Date
Fredrik de Groot
303bd963d5 manifest: add optional base check on remove and extend
This adds an optional, built-in checker for
guarding against patches hanging on wrong
base revisions, which is useful if a lower layer of
the manifest changes after a patch was done.

When adding a patch with a new revision using
extend-project or remove-project/project:

          C---D---E patches in project bla
         /
    A---B project bla in manifest state 1

<extend-project name="bla" revision="E" base-rev="B">

If project bla gets updated, in a new snap ID
or by a supplier or similar, to a new state:

          C---D---E patches in project bla
         /
    A---B---F---G project bla in manifest state 2

Parsing will fail because revision of bla is now G,
giving the choice to create a new patch branch
from G and updating base-rev, or keeping previous
branch for some reason and only updating base-rev.

Intended for use in a layered manifest with
hashed revisions. Named refs like branches and tags
also work fine when comparing, but will be misleading
if a branch is used as base-rev.

Change-Id: Ic6211550a7d3cc9656057f6a2087c505b40cad2b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/436777
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Fredrik de Groot <fredrik.de.groot@haleytek.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-10-28 16:55:10 +00:00
Josip Sokcevic
ae384f8623 [event_log] Stop leaking semaphore resources
With the global state and fork, we are left with uncleaned resources.
Isolate mulitprocessing.Value in a function so we stop the leak.

Bug: 353656374
Change-Id: If50bb544bda12b72f00c02bc1d2c0d19de000b88
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440261
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
2024-10-24 16:58:17 +00:00
Kuang-che Wu
70a4e643e6 progress: always show done message
The done message was omitted if the task is shorter than 0.5s. This
might confuse users.

Bug: b/371638995
Change-Id: I3fdd2cd8daea16d34fba88457d09397fff71af15
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440222
Tested-by: Kuang-che Wu <kcwu@google.com>
Commit-Queue: Kuang-che Wu <kcwu@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
2024-10-24 16:21:28 +00:00
Kuang-che Wu
8da4861b38 subcmds: reduce multiprocessing serialization overhead
Follow the same approach as 39ffd9977e to reduce serialization overhead.

Below benchmarks are tested with 2.7k projects on my workstation
(warm cache). git tracing is disabled for benchmark.

(seconds)              | v2.48 | v2.48 | this CL | this CL
	               |       |  -j32 |         |    -j32
-----------------------------------------------------------
with clean tree state:
branches (none)        |   5.6 |   5.9 |    1.0  |    0.9
status (clean)         |  21.3 |   9.4 |   19.4  |    4.7
diff (none)            |   7.6 |   7.2 |    5.7  |    2.2
prune (none)           |   5.7 |   6.1 |    1.3  |    1.2
abandon (none)         |  19.4 |  18.6 |    0.9  |    0.8
upload (none)          |  19.7 |  18.7 |    0.9  |    0.8
forall -c true         |   7.5 |   7.6 |    0.6  |    0.6
forall -c "git log -1" |  11.3 |  11.1 |    0.6  |    0.6

with branches:
start BRANCH --all     |  21.9 |  20.3 |   13.6  |    2.6
checkout BRANCH        |  29.1 |  27.8 |    1.1  |    1.0
branches (2)           |  28.0 |  28.6 |    1.5  |    1.3
abandon BRANCH         |  29.2 |  27.5 |    9.7  |    2.2

Bug: b/371638995
Change-Id: I53989a3d1e43063587b3f52f852b1c2c56b49412
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/440221
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Kuang-che Wu <kcwu@google.com>
Commit-Queue: Kuang-che Wu <kcwu@google.com>
2024-10-23 23:34:34 +00:00
Kuang-che Wu
39ffd9977e sync: reduce multiprocessing serialization overhead
Background:
 - Manifest object is large (for projects like Android) in terms of
   serialization cost and size (more than 1mb).
 - Lots of Project objects usually share only a few manifest objects.

Before this CL, Project objects were passed to workers via function
parameters. Function parameters are pickled separately (in chunk). In
other words, manifests are serialized again and again. The major
serialization overhead of repo sync was
  O(manifest_size * projects / chunksize)

This CL uses following tricks to reduce serialization overhead.
 - All projects are pickled in one invocation. Because Project objects
   share manifests, pickle library remembers which objects are already
   seen and avoid the serialization cost.
 - Pass the Project objects to workers at worker intialization time.
   And pass project index as function parameters instead. The number of
   workers is much smaller than the number of projects.
 - Worker init state are shared on Linux (fork based). So it requires
   zero serialization for Project objects.

On Linux (fork based), the serialization overhead is
  O(projects)  --- one int per project
On Windows (spawn based), the serialization overhead is
  O(manifest_size * min(workers, projects))

Moreover, use chunksize=1 to avoid the chance that some workers are idle
while other workers still have more than one job in their chunk queue.

Using 2.7k projects as the baseline, originally "repo sync" no-op
sync takes 31s for fetch and 25s for checkout on my Linux workstation.
With this CL, it takes 12s for fetch and 1s for checkout.

Bug: b/371638995
Change-Id: Ifa22072ea54eacb4a5c525c050d84de371e87caa
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439921
Tested-by: Kuang-che Wu <kcwu@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Kuang-che Wu <kcwu@google.com>
2024-10-23 02:58:45 +00:00
Kaushik Lingarkar
584863fb5e Fix incremental syncs for prjs with submodules
When performing an incremental sync (re-running repo init with an
updated manifest revision) with --fetch-submodules or sync-s=true,
there is an attempt to get a list of all projects (including
submodules) before projects are actually fetched. However, we can
only list submodules of a project if we have already fetched its
revision. Instead of throwing an error when we don't have the
revision, assume there are no submodules for that project. In the
sync cmd, we already update the list of projects to include
submodules after fetching superprojects.

Change-Id: I48bc68c48b5b10117356b18f5375d17f9a89ec05
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/439761
Commit-Queue: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
Tested-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Nasser Grainawi <nasser.grainawi@linaro.org>
2024-10-18 03:55:10 +00:00
Josip Sokcevic
454fdaf119 sync: Always use WORKER_BATCH_SIZE
With 551285fa35, the comment about number
of workers no longer stands - dict is shared among multiprocesses and
real time information is available.

Using 2.7k projects as the baseline, using chunk size of 4 takes close
to 5 minutes. A chunk size of 32 takes this down to 40s - a reduction of
rougly 8 times which matches the increase.

R=gavinmak@google.com

Bug: b/371638995
Change-Id: Ida5fd8f7abc44b3b82c02aa0f7f7ae01dff5eb07
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438523
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2024-10-07 18:44:19 +00:00
Josip Sokcevic
f7f9dd4deb project: Handle git sso auth failures as repo exit
If a user is not authenticated, repo continues execution and it will
likely result in more of the same errors being printed. A user is also
likely to SIGTERM the process resulting in more errors.

This change stops repo sync if any of repositories can't be fetched to
Git authentcation using sso helper. We could extend this to all Git
authentication

Change-Id: I9e471e063450c0a51f25a5e7f12a83064dfb170c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438522
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-10-03 20:47:50 +00:00
Josip Sokcevic
70ee4dd313 superproject: Remove notice about beta
It's been the default for Android for over a year now, and it's no
longer useful notice.

Change-Id: I53c6f1e7cee8c1b2f408e67d3a6732db3b272bee
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438521
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-10-03 20:37:18 +00:00
Josip Sokcevic
cfe3095e50 project: run fetch --refetch on unable to not parse commit
Similarly to e59e2ae757, handle missing
gc'ed commits by running `git fetch --refetch`.

R=jojwang@google.com

Bug: b/360889369
Bug: b/371000949
Change-Id: I108b870b855d3b9f23665afa134c6e35f7cd2830
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438461
Reviewed-by: Joanna Wang <jojwang@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-10-03 17:40:37 +00:00
Josip Sokcevic
621de7ed12 Disable git terminal prompt during fetch/clone
git fetch operation may prompt user to enter username and password.
This won't be visible to user when repo sync operation since stdout and
stderr are redirected. If that happens, user may think repo is doing
work and likely won't realize it's stuck on user's input.

This patch disables prompt for clone and fetch operations, and repo will
fail fast.

R=gavinmak@google.com

Bug: b/368644181
Change-Id: I2efa88ae66067587a00678eda155d861034b9127
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/438001
Reviewed-by: Nasser Grainawi <nasser.grainawi@linaro.org>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2024-09-26 22:10:36 +00:00
Kaushik Lingarkar
d7ebdf56be init: add --manifest-upstream-branch
When a sha1 is provided to '--manifest-branch', the ref which
is expected to contain that sha1 can be provided using the new
'--manifest-upstream-branch' option. This is useful with
'--current-branch' to avoid having to sync all heads and tags,
or with a commit that comes from a non-head/tag ref (like a
Gerrit change ref).

Change-Id: I46a3e255ca69ed9e809039e58b0c163e02af94ef
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/436717
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
Tested-by: Kaushik Lingarkar <kaushik.lingarkar@linaro.org>
2024-09-26 00:52:28 +00:00
Kaushik Lingarkar
fabab4e245 man: regenerate man pages
Change-Id: Icf697eda7d5dcdc87854ad6adf607353c7ba5ac2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/437941
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Nasser Grainawi <nasser.grainawi@linaro.org>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
2024-09-25 20:57:42 +00:00
Brian Norris
b577444a90 project: Copy and link files even with local branches
In the winding maze that constitutes Sync_LocalHalf(), there are paths
in which we don't copy-and-link files. Examples include something like:

  cd some/project/
  repo start head .
  # do some work, make some commit, upload that commit to Gerrit

  [[ ... in the meantime, someone addes a <linkfile ...> for
     some/project/ in the manifest ... ]]

  cd some/project/
  git pull --rebase
  repo sync

In this case, we never hit a `repo rebase` case, which might have saved
us. Instead, the developer is left confused why some/project/ never had
its <linkfile>s created.

Notably, this opens up one more corner case in which <linkfile ... /> or
<copyfile ... /> could potentially clobber existing work in the
destination directory, but there are existing cases where that's true,
and frankly, those seem like bigger holes than this new one.

Change-Id: I394b0e4529023a8ee319dc25d03d513a19251a4a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/437421
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Brian Norris <briannorris@google.com>
Commit-Queue: Brian Norris <briannorris@google.com>
2024-09-19 00:11:52 +00:00
Navil
1e19f7dd61 sync: include TARGET_RELEASE when constructing smart sync target.
When using the smart sync option, we try to construct the target that
was "lunched" from the TARGET_PRODUCT and TARGET_BUILD_VARIANT envvars.

However, an android target is now made of three parts,
{TARGET_PRODUCT}-{TARGET_RELEASE}-{TARGET_BUILD_VARIANT}.

I am leaving the option of creating a target if a TARGET_RELEASE is not
specified in case there are other consumers who depend on that option.

BUG=b:358101714
TEST=./run_tests
TEST=smart sync on android repo and manually inspecting
smart_sync_override.xml

Change-Id: I556137e33558783a86a0631f29756910b4a93d92
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/436977
Tested-by: Yiwei Zhang <yiwzhang@google.com>
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Yiwei Zhang <yiwzhang@google.com>
2024-09-12 16:15:50 +00:00
bright
d8b4101eae color: fix have_fg not re assign to true
In method _parse the value of this variable 'have_fg ' is always
False, Maybe reassign it to True is lost.
I guess the author’s original intention was:
if set some value in gitconfig file(for ex: text = black red ul),
the first is bg color, the second is fg color, and the last one is attr.



Change-Id: I372698fe625db4c1fdaa94ea7f193a80a850ecb9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/425997
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Bright Ma <mmh1989@foxmail.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-09-12 16:15:06 +00:00
Peter Kjellerstedt
1c53b0fa44 tox.ini: Make the lint and format environments run black for all code
This matches the extra files specified in run_tests.

Change-Id: Ic8999383a17b3ec7ae27322323ea44eeaa40c968
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/434998
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-09-12 16:09:24 +00:00
Peter Kjellerstedt
e5ae870a2f tox.ini, constraints.txt: Lock the version of black to <24
The formatting produced by black versions before 24 matches the current
formatting of the code.

Change-Id: I045f22d2f32a09d4683867293e81512f2abd1036
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/434997
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-09-12 16:05:35 +00:00
Josip Sokcevic
e59e2ae757 project: run fetch --refetch onacould not parse commit
git may gc reachable objects in partial clone repository due to a bug
(report:
https://lore.kernel.org/git/20240802073143.56731-1-hanyang.tony@bytedance.com/
). Until git is properly patched and released, force --refetch iff
"could not parse commit" is part of git output. --refetch will will
ensure that gc'ed objects are retrieved.

Bug: b/360889369
Change-Id: I0fc911c591060f859235dcd8d019881106f0858e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/437017
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Sam Saccone <samccone@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
2024-09-12 15:27:12 +00:00
Jeroen Dhollander
c44ad09309 Add a --rebase option to sync command
Previously repo would abort a sync if there were published changes not
merged upstream. The --rebase option allows the modification of
published commits.

This is a copy of http://go/grev/369694 with the merge conflicts
resolved.

Bug: 40014610
Change-Id: Idac8199400346327b530abea33f1ed794e5bb4c2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/435838
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Jeroen Dhollander <jeroendh@google.com>
Commit-Queue: Jeroen Dhollander <jeroendh@google.com>
2024-08-30 09:08:29 +00:00
Erik Elmeke
4592a63de5 sync: Fix git command for aborting rebase being called incorrectly.
The argument list was incorrectly destructured so only the first
element of the list was considered a git-cmd, split up by each
character in the string.

Change-Id: Idee8a95a89a7da8b8addde07135354fc506c2758
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/435839
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Erik Elmeke <erik@haleytek.corp-partner.google.com>
Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com>
2024-08-28 08:56:35 +00:00
Mike Frysinger
0444ddf78e project: ignore more curl failure modes
Current clone bundle fetches from Google storage results HTTP/404
and curl exiting 56.  This is basically WAI, so stop emitting
verbose error output whenever that happens.  Also add a few more
curl exit statuses based on chromite history, and document them.

Change-Id: I3109f8a8a19109ba9bbd62780b40bbcd4fce9b76
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/432197
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2024-07-02 19:03:54 +00:00
Shik Chen
9bf8236c24 logging: Fix log formatting with colored output
The log message is already formatted before being passed to the colorer.
To avoid the exception "TypeError: not enough arguments for format
string", we should use the `nofmt_colorer` instead.

This bug occurs only when the formatted string still contains '%'
character. The following snippet can reproduce the bug:

```
from repo_logging import RepoLogger
RepoLogger(__name__).error("%s", "100% failed")
```

Change-Id: I4e3977b3d21aec4e0deb95fc1c6dd1e59272d695
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/432017
Tested-by: Shik Chen <shik@google.com>
Commit-Queue: Shik Chen <shik@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2024-07-02 06:24:31 +00:00
Mike Frysinger
87f52f308c upload: add a --topic option for setting topic explicitly
Let people specify the exact topic when uploading CLs.  The existing
-t option only supports setting the topic to the current local branch.

Add a --topic-branch long option to the existing -t to align it a bit
better with --hashtag & --hashtag-branch.

Change-Id: I010abc4a7f3c685021cae776dd1e597c22b79135
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/431997
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-07-01 17:54:19 +00:00
Erik Elmeke
562cea7758 sync: Abort rebase in progress if force-checkout is set
This will make "repo sync -d --force-checkout" more reliable
in CI automation, as there are fewer things in the way that may
need manual intervention.

Bug: b/40015382
Change-Id: I8a79971724a3d9a8e2d682b7a0c04deda9e34177
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/423317
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com>
Commit-Queue: Erik Elmeke <erik@haleytek.corp-partner.google.com>
2024-05-23 14:14:18 +00:00
Erik Elmeke
eede374e3e ssh: Set git protocol version 2 on SSH ControlMaster
According to https://git-scm.com/docs/protocol-v2#_ssh_and_file_transport,
when using SSH, the environment variable GIT_PROTOCOL must be set
when establishing the connection to the git server.

Normally git does this by itself. But in repo-tool where the SSH
connection is managed by the repo-tool, it must be passed in
explicitly instead.

Under some circumstances of environment configuration, this
caused all repo sync commands over ssh to always use
git protocol version 1. Even when git was configured to use
version 2.

Using git protocol v2 can significantly improve fetch speeds,
since it uses server side filtering of refs, reducing the
amount of unneccessary objects to send.

Change-Id: I6d4c3b7300a6090d707480b1a638ed03622fa71a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/411362
Tested-by: Erik Elmeke <erik@haleytek.corp-partner.google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Erik Elmeke <erik@haleytek.corp-partner.google.com>
2024-05-16 13:26:46 +00:00
Mike Frysinger
2c5fb84d35 upload: drop check for uncommitted local changes
git push, like most git commands, does not warn or otherwise prompt
users when there are local uncommitted changes.  To simplify the
upload logic, and to harmonize repo upload with git push as a more
git-esque flow, stop checking/warning/prompting the user here too.

Change-Id: Iee18132f0faad0881f1a796cb58821328e04b694
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/425337
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-05-14 02:32:27 +00:00
Mike Frysinger
12f6dc49e9 git: raise hard version to 1.9.1
Debian 7 Wheezy went EOL in May 2018.  We don't need to carry support
for that anymore as there have been 5 major releases since.  Ubuntu
Precise went EOL in Apr 2019 (including the extended support phase).
That means we can bump the required git version from 1.7.9 to 1.9.1.

git-1.7.9 was released in 2012 while git-1.9.1 was released in 2014.
So that shouldn't be a problem either.  And we've been warning people
using git versions older than 1.9.1 for 3 years now that they need to
upgrade.

Change-Id: Ifbbf72f51010b0a944c2785895d1b605333f9146
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415637
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-05-01 15:23:50 +00:00
Mike Frysinger
5591d99ee2 release: update-hooks: helper for automatically syncing hooks
These hooks are maintained in other projects.  Add a script to automate
their import so people don't send us changes directly, and we can try to
steer them to the correct place.

Change-Id: Iac0bdb3aae84dda43a1600e73107555b513ce82b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/422177
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
2024-04-23 18:31:51 +00:00
Mike Frysinger
9d865454aa gitc: delete a few more dead references
Change-Id: I1da6f2ee799c735a63ac3ca6e5abd1211af10433
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/419217
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-04-18 02:30:06 +00:00
Mike Frysinger
cbd78a9194 man: regenerate man pages
Change-Id: I8d9dcb37f315d4208b7c8005206ae939dad79a3e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/419197
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
2024-04-18 02:28:33 +00:00
Kaiyi Li
46819a78a1 Remove platform_utils.realpath
... since it's just a simple wrapper of os.path.realpath now.

Change-Id: I7433e5fe09c64b130f06e2541151dce1961772c9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/416637
Tested-by: Kaiyi Li <kaiyili@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Kaiyi Li <kaiyili@google.com>
2024-03-27 17:13:58 +00:00
Kaiyi Li
159389f0da Fix drive mounted directory on Windows
On my Windows machine, I mount drive D: to the directory C:\src.

The old implementation returns the incorrect 'C:\\??\\Volume{ad2eb15e-f293-4d48-a448-54757d95a97c}' result, which breaks the repo init command.

With the use of os.path.realpath, it can return 'D:\\' correctly.

Change-Id: Ia5f53989055125cb282d4123cf55d060718aa1ff
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/416580
Reviewed-by: Greg Edelston <gredelston@google.com>
Tested-by: Kaiyi Li <kaiyili@google.com>
Commit-Queue: Kaiyi Li <kaiyili@google.com>
2024-03-27 14:00:47 +00:00
Mike Frysinger
4406642e20 git_command: unify soft/hard versions with requirements.json
Use the requirements logic in the wrapper to load versions out of the
requirements.json file to avoid duplicating them in git_command.py.

Change-Id: Ib479049fc54ebc6f52c2c30d1315cf1734ff1990
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415617
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2024-03-21 21:20:50 +00:00
Josip Sokcevic
73356f1d5c project: Check if dotgit exists w/out symlink check
os.path.exists returns false on a broken symlink. This is not what repo
needs when checking if a project is setup properly.

For example, if src/foo/.git can't be resolved, repo tries to create
symlink and that results in FileExistsError.

Use lexists which returns True even if symlink is broken. That will
force path where repo checks where symlink is pointing to and will fix
it to the correct location.

Bug: b/281746795
Change-Id: Id3f7dc3a3cb6499d02ce7335eca992ddc7deb645
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415197
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: George Engelbrecht <engeg@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
2024-03-20 22:09:14 +00:00
Mike Frysinger
09fc214a79 git: raise soft version to 2.7.4
git-1.9.1 was released in 2014 while git-2.7.4 was released in 2016.
Debian Stretch was released in 2017 and Ubuntu Xenial was released in
2016 which are plenty old at this point.  Both of those include at
least git-2.7.4.

We will start warning users of Debian Jessie (released in 2015 & EOL
in 2020) and Ubuntu Trusty (released in 2014 & EOL Apr 2024) that
they will need to upgrade.

Change-Id: I6be3809bc45968fdcb02cad3f7daf75ded1bb5b1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415137
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-03-20 21:11:26 +00:00
Mike Frysinger
3762b17e98 git: raise hard version to 1.7.9
Debian 6 Squeeze went EOL in Feb 2016.  We don't need to carry support
for that anymore as there have been 6 major releases since.  That means
we can bump the required git version from 1.7.2 to 1.7.9.  Ubuntu Precise
shipped with the latter.

git-1.7.2 was released in 2010 while git-1.7.9 was released in 2012.
So that shouldn't be a problem either.  And we've been warning people
using git versions older than 1.9.1 for 3 years now that they need to
upgrade.

Change-Id: I7712f110ea158297b489b8379b112c6700b21a46
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415097
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-03-20 19:49:44 +00:00
Mike Frysinger
ae419e1e01 docs: release: add recent git/python/ssh/debian info
Change-Id: I744360b1bfc816e94b3511f0130abb2c08dedd42
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/415117
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Mike Frysinger <vapier@google.com>
2024-03-20 19:49:01 +00:00
Josip Sokcevic
a3a7372612 main: Stringify project name in error_info
If a project can't be removed from checkout due to uncommitted changes
present, error.project is type of Project and not a string (as it is in
some cases). Project is not JSON serializable, resulting in exception
within exception handler:

TypeError: Object of type Project is not JSON serializable

This change casts project to string as a defensive mechanism. It also
passes project name instead of project object.

Change-Id: Ie7b782d73dc3647975755d5a3774d16ea6cd5348
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/413877
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-03-15 19:26:10 +00:00
Sebastian Schuberth
fff1d2d74c ssh: Print details if running the command fails
Change-Id: I87adbdd5fe4eb2709c97ab4c21b414145acf788b
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/392915
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Tuan Vo Hung <vohungtuan@gmail.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-03-11 16:40:55 +00:00
Josip Sokcevic
4b01a242d8 upload: Fix patchset description destination
Bug: 308467447
Change-Id: I8ad598d39f5fdb24d549d3277ad5fedac203581b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/412477
Reviewed-by: George Engelbrecht <engeg@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-03-08 18:05:36 +00:00
Josip Sokcevic
46790229fc sync: Fix sorting for nested projects
The current logic to create checkout layers doesn't work in all cases.
For example, let's assume there are three projects: "foo", "foo/bar" and
"foo-bar". Sorting lexicographical order is incorrect as foo-bar would
be placed between foo and foo/bar, breaking layering logic.

Instead, we split filepaths based using path delimiter (always /) and
then use lexicographical sort.

BUG=b:325119758
TEST=./run_tests, manual sync on chromiumos repository

Change-Id: I76924c3cc6ba2bb860d7a3e48406a6bba8f58c10
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/412338
Tested-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: George Engelbrecht <engeg@google.com>
2024-03-08 17:58:24 +00:00
Josip Sokcevic
edadb25c02 sync: introduce --force-checkout
In some cases (e.g. in a CI system), it's desirable to be able to
instruct repo to force checkout. This flag passes --force flag to `git
checkout` operations.

Bug: b/327624021
Change-Id: I579edda546fb8147c4e1a267e2605fcf6e597421
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/411518
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: George Engelbrecht <engeg@google.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
2024-03-07 17:21:51 +00:00
Sergiy Belozorov
96edb9b573 upload: Add support for setting patchset description
Bug: 308467447
Change-Id: I7abcbc98131b826120fc9ab85d5b889f90db4b0a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/355968
Tested-by: Sergiy Belozorov <sergiyb@chromium.org>
Reviewed-by: Mike Frysinger <vapier@google.com>
Commit-Queue: Sergiy Belozorov <sergiyb@chromium.org>
2024-03-04 18:50:24 +00:00
Josip Sokcevic
5554572f02 sync: Introduce git checkout levels
If a repo manifest is updated so that project B is placed within a
project A, and if project A had content in new B's location in the old
checkout, then repo sync could break depending on checkout order, since
B can't be checked out before A.

This change introduces checkout levels which enforces right sequence of
checkouts while still allowing for parallel checkout. In an example
above, A will always be checked out first before B.

BUG=b:325119758
TEST=./run_tests, manual sync on ChromeOS repository

Change-Id: Ib3b5e4d2639ca56620a1e4c6bf76d7b1ab805250
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410421
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Greg Edelston <gredelston@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
2024-02-27 17:28:33 +00:00
Peter Collingbourne
97ca50f5f9 git_command: Return None from GetEventTargetPath() if set to empty string
If trace2.eventTarget was set to the empty string,
match git behavior and don't write a trace.

Bug: 319673783
Change-Id: I02b3884ad97551f8a9d7363c2cbe6b0adee6f73e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410518
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Tested-by: Peter Collingbourne <pcc@google.com>
2024-02-26 17:51:11 +00:00
Josip Sokcevic
8896b68926 trace: Save trace2 sid in REPO_TRACE file
git-trace2 events contain additional information what git is doing under
the hood, and repo doesn't have visibility into.

Instead of relying on timestamp information to match REPO_TRACE with
git-trace2 events, add SID information into REPO_TRACE.

Change-Id: I37672a3face81858072c7a3ce34ca3379199dab5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410280
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-02-22 20:55:09 +00:00
Greg Edelston
fec8cd6704 subcmds: sync: Remove deprecated _AUTO_GC
Opportunistic cleanup. It looks like this deprecated feature was slated
for deletion nearly a year ago.

Bug: None
Test: ./run_tests
Change-Id: I0bd0c0e6acbd1eaee1c0b4945c79038257d22f44
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/410198
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Greg Edelston <gredelston@google.com>
Tested-by: Greg Edelston <gredelston@google.com>
2024-02-20 19:55:15 +00:00
Josip Sokcevic
b8139bdcf8 launcher: Set shebang to python3
Some (most?) Linux repos don't have /usr/bin/python, unless
python-is-python3 is installed. While package owners can adjust shebang,
we have seen an increase in number of bugs filed as extra steps are
required.

Per PEP-0394, python3 is acceptable and should be available if python3
is supported. We no longer support python2, and repo no longer works
with python2, so this change makes that explicit.

Bug: 40014585
Change-Id: I9aed90fd470ef601bd33bd596af3df69da69ef5d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/407497
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Jason Chang <jasonnc@google.com>
Reviewed-by: Gavin Mak <gavinmak@google.com>
Reviewed-by: Максим Паймушкин <maxim.paymushkin@gmail.com>
Tested-by: Josip Sokcevic <sokcevic@google.com>
2024-02-07 20:44:32 +00:00
Jason Chang
26fa3180fb sync: ensure RepoChangedException propagated
Prior to this change RepoChangedException would be caught and re-rasied
as a different exception. This would prevent RepoChangedException
handler from running in main.py

Bug: b/323232806
Change-Id: I9055ff95d439d6ff225206c5bf1755cc718bcfcc
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/407144
Tested-by: Josip Sokcevic <sokcevic@google.com>
Reviewed-by: Josip Sokcevic <sokcevic@google.com>
Commit-Queue: Josip Sokcevic <sokcevic@google.com>
2024-02-06 18:46:19 +00:00