Commit Graph

381 Commits

Author SHA1 Message Date
Samuel Holland
5f0e57d2ca init: Remove string concat in no-op os.path.join
This also fixes a line length warning.

Change-Id: I9c1ab65f83a35581dd657a707c7bc3c69db2b1dc
2018-01-22 11:00:24 -06:00
Samuel Holland
baa0009355 Support relative paths in --reference
Put the correctly-expanded relative paths in objects/info/alternates.
From gitrepository-layout(5), this path should be "relative to the
object database, not to the repository".

Change-Id: I7b2027ae23cf7d367b80f5a187603c4cbacdb2de
2018-01-22 10:57:29 -06:00
Masaya Suzuki
305a2d029f Support --push-option in upload subcommand
Change-Id: I44836f8c66ded5a96cbf5431912e027e681f6529
2017-11-13 15:48:49 -08:00
David Pursehouse
f46902a800 forall: Clarify expansion of REPO_ environment values with -c
If a user executes:

  repo forall -c echo $REPO_PROJECT

then $REPO_NAME is expanded by the user's shell first, and passed
as $1 to the shell that executes echo. This will either result in
no output, or output of whatever REPO_NAME is set to in the user's
shell. Either way, this is an unexpected result.

The correct way to do it is:

  repo forall -c 'echo $REPO_PROJECT'

such that $REPO_NAME is passed in to the shell literally, and then
expanded to the value set in the environment that was passed to
the shell.

Update the documentation to make this clearer.

Change-Id: I713caee914172ad8d8f0fafacd27026502436f0d
2017-10-31 13:07:55 +09:00
Renaud Paquay
010fed7711 Replace all os.remove calls
os.remove raises an exception when deleting read-only files on
Windows. Replace all calls with calls to platform_utils.remove,
which deals with deals with that issue.

Change-Id: I4dc9e0c9a36b4238880520c69f5075eca40f3e66
2017-08-31 13:49:36 -07:00
Renaud Paquay
227ad2ef42 Implement islink, readlink and realpath using Win32 api
Change-Id: I18452cbb32d24db73601ad10485dbe6bb278731c
2017-08-31 13:49:01 -07:00
David Pursehouse
2c57d619bc Merge "Add option '--no-cert-checks' for 'upload' sub command." 2017-08-30 15:11:10 +00:00
David Pursehouse
d1ebc89a08 Merge changes from topic "windows-support"
* changes:
  Port os.rename calls to work on Windows
  Workaround shutil.rmtree limitation on Windows
  Add support for creating symbolic links on Windows
  Make "git command" and "forall" work on Windows
2017-08-30 10:24:03 +00:00
Zac Livingston
9ead97bb51 When starting a branch, do not use a tag or change value for branch.merge
When starting a branch, branch.merge is set to project revision unless
the revision is a SHA1. In that case, branch.merge is set to dest_branch
if defined or manifest default revision otherwise. This special handling
allows repo upload to work when the project revision is a SHA1.

Extend the special handling to also happen when the project revision
is a tag value or a change value so that repo upload will work in those
case as well.

Change-Id: Iff81ece40e770cd02535e80dcb023564d42dcf47
2017-08-25 09:10:29 +09:00
Łukasz Gardoń
bed59cec5e Add option '--no-cert-checks' for 'upload' sub command.
This option allow to bypass verification ssl certification while
establishing connection with Gerrit to upload review.

Change-Id: If2e15f5a273c18a700eb5093ca8a4d5a4cbf80cd
2017-08-23 14:06:14 +02:00
Jonathan Nieder
c94d6eb902 Revert "Migrate git-repo to create private changes rather than drafts"
This reverts commit d88f53e2b9. I merged
it too hastily without paying enough attention to compatibility with
released Gerrit versions.

Change-Id: I4028d4737df1255f11e217da183a19a010597d5b
2017-08-08 18:34:53 +00:00
Changcheng Xiao
d88f53e2b9 Migrate git-repo to create private changes rather than drafts
Considering that some users might expect changes created with
'-d' option are not public. Private changes may be a better
choice here than work-in-progress changes.

Change-Id: I46a8fb9ae38beb41cf96d6abe82bea6db2439669
2017-08-07 15:08:18 +02:00
Changcheng Xiao
87984c6db4 Add options for git-repo to support private and wip changes
This change adds options for git-repo tool to support private
changes and work-in-progress changes.

Change-Id: I343491f5949f06f1580d53f9cc0dee2dca09130f
2017-08-07 15:02:39 +02:00
David Pursehouse
ffc1401327 Merge "download: try to choose . as default project if none" 2017-08-02 07:19:45 +00:00
David Pursehouse
8a6eeed7f5 Merge "Always print percentage when syncing quietly" 2017-08-02 07:01:08 +00:00
Tim Schumacher
7be072efa6 Always print percentage when syncing quietly
Change-Id: I574396e63520781067ed1e991c41caf7640e5731
2017-07-15 16:44:55 +00:00
Nicolas Cornu
7482a96443 download: try to choose . as default project if none
Change-Id: I28b5e3be5f3c9a4c077af87d6a3e0cc3b96a1b9d
2017-07-12 10:15:06 +02:00
David Pursehouse
3bcd30545e Fix "list comprehension redefines 'x'" warnings from pyflakes
$ git ls-files | grep py$ | xargs pyflakes
  subcmds/stage.py:101: list comprehension redefines 'p' from line 63
  subcmds/sync.py:784: list comprehension redefines 'p' from line 664
  subcmds/upload.py:467: list comprehension redefines 'avail' from line 454

Change-Id: Ia65d1a72ed185ab3357e1a91ed4450c719e75a7c
2017-07-10 23:26:04 +00:00
Martin Kelly
224a31a765 init: add missing submodule arg
The submodule argument to Sync_LocalHalf was missing in
MetaBranchSwitch, causing submodules not to get synced when the
-b/--manifest-branch argument to init is used.

Change-Id: Ie86d271abac2020725770be36ead83be3326e64b
Signed-off-by: Martin Kelly <mkelly@xevo.com>
2017-07-10 14:50:52 -07:00
Nicolas Cornu
b54343d9fd Tell the user if it will upload a draft
Change-Id: Ie004ec9d61603f3f618c47597947b82c59f2839c
2017-07-10 13:20:01 +00:00
David Pursehouse
259f16520a Merge "Add a newline after "Fetching projects" progress output" 2017-06-28 04:24:03 +00:00
Nicolas Cornu
8419ab22d6 sync: Continue job if some fetchs failed but force-broken is set
With --force-broken it continue to fetch other projects but nothing
is added in directory because it abort some lines later.

Change-Id: I32c4a4619b3028893dc4f98e8d4e5bc5c09adb27
2017-06-16 12:23:26 +02:00
Tim Schumacher
913327f10c
Add a newline after "Fetching projects" progress output
Output before change:

    Fetching project platform/packages/providers/UserDictionaryProvider
    Fetching projects:  66% (773/1171)  Fetching project platform/external/regex-re2
    Fetching project device/generic/mini-emulator-x86_64

Output after change:

    Fetching project platform/packages/providers/UserDictionaryProvider
    Fetching projects:  66% (773/1171)
    Fetching project platform/external/regex-re2
    Fetching project device/generic/mini-emulator-x86_64

Change-Id: I4da84da58316c69294e4da2792f83885dc942701
2017-06-13 13:03:39 +02:00
Renaud Paquay
a65adf74f9 Workaround shutil.rmtree limitation on Windows
By default, shutil.rmtree raises an exception when deleting readonly
files on Windows.

Replace all shutil.rmtree with platform_utils.rmtree, which adds an
error handler to make files read-write when they can't be deleted.

Change-Id: I9cfea9a7b3703fb16a82cf69331540c2c179ed53
2017-05-29 19:32:31 +09:00
Renaud Paquay
2e70291162 Make "git command" and "forall" work on Windows
Python on Windows does not support non blocking file operations.
To workaround this issue, we instead use Threads and a Queue to
simulate non-blocking calls. This is happens only when running
with the native Windows version of Python, meaning Linux and Cygwin
are not affected by this change.

Change-Id: I4ce23827b096c5138f67a85c721f58a12279bb6f
2017-05-29 19:29:30 +09:00
David Pursehouse
a24671f661 Merge "init: allow relative path on --reference argument" 2017-05-29 04:54:12 +00:00
David Riley
e0684addee sync: Add support to dump a JSON event log of all sync events.
Change-Id: Id4852968ac1b2bf0093007cf2e5ca951ddab8b3b
2017-05-29 13:39:54 +09:00
YOUNG HO CHA
c79d3b8fd1 init: allow relative path on --reference argument
Change-Id: I41d6be6bc035fdddb5a27c072994439986d58d58
Signed-off-by: YOUNG HO CHA <ganadist@gmail.com>
2017-05-28 00:33:25 +00:00
David Pursehouse
fbcbcabe98 Merge "init: add --submodules to sync manifest submodules" 2017-05-27 04:24:58 +00:00
David Pursehouse
666debc518 gitc_delete: Remove unused imports
Change-Id: I672189ba99e18dca3956e2396c921d1ef0ca2ddd
2017-05-26 21:53:34 +09:00
David Pursehouse
c354a9b922 abandon: fix usage of undefined variable
As reported by pyflakes:

  subcmds/abandon.py:84: undefined name 'p'

The name of the variable should be 'proj'.

Change-Id: Ic09eb92e8db6b510e99efce010bd0bb094d7cbfe
2017-05-26 21:52:23 +09:00
Martin Kelly
e4e94d26ae init: add --submodules to sync manifest submodules
repo sync can sync submodules via the --fetch-submodules option.
However, if the manifest repo has submodules, those will not be synced.
Having submodules in the manifest repo -- while not commonly done -- can
be useful for inheriting a manifest from another project using <include>
and layering changes on top of it.  In this way, you can avoid having to
deal with merge conflicts between your own manifests and the other
project's manifests (for example, if you're managing an Android fork).

Add a --submodule option to init that automatically syncs the submodules
in the manifest repo whenever the manifest repo changes.

Change-Id: I45d34f04517774c1462d7f233f482d1d81a332a8
Signed-off-by: Martin Kelly <mkelly@xevo.com>
2017-05-23 16:51:31 -07:00
Marc Herbert
ffb4b89099 sync.py: report the remote URL on fatal git remote errors
repo can be configured to download from any number of remote git repos.
However when one fails repo doesn't report which one. Example:
Fatal: remote error: Daily ls-remote rate limit exceeded for IP xx.xx.xx.xx

TEST=repo init -q -u https://chromium.googlesource.com/chromiumos/manifest.git
  # Apply patch in ./.repo/repo/
  # Simulate a git remote error:
  sed -i -e 's#chromiumos/docs#chromiumos/XXdocs#' .repo/manifests/full.xml
  repo sync --quiet --force-sync docs
  # error message now shows the remote URL

Optional test tip: reduce the time.sleep(random(...)) in ./.repo/repo/project.py

Change-Id: I4509383b6a43a8e66064778e8ed612d8a735c8b6
2017-04-04 22:10:34 -07:00
Naseer Ahmed
f4dda9a1be init: Add no-tags and current branch options
This avoids fetching tags and branches for huge manifests

Change-Id: I19c9724d75364440b881b297d42b906f541f73ff
2016-12-01 19:03:41 -05:00
David Pursehouse
ae81c964b6 Merge "implement optional '--all' in the abandon command" 2016-10-28 07:52:31 +00:00
Andrew Wheeler
4d5bb68d58 status: add -q/--quiet option
The --quiet option reduces the output to just
a list of projects with modified workspaces (and
orphans if -o is specified)

A common use case is when performing a full-workspace
merge.  The integrator will kick-off a merge via:

    repo forall -c git merge <some tag>

And then produce a short list of conflicted projects via:

    repo status -q

The integrator can then iteratively fix and clean up all conficted
components.  The merge is complete when:

    repo status -q

    returns no output.

Change-Id: Ibbba8713eac35befd8287c95948874e23fd5c7e2
2016-10-17 15:24:09 -05:00
Kyunam.jo
2e14792a94 implement optional '--all' in the abandon command
when you want to delete all local branches, you should be find
all branches' name, and type them behind 'repo abandon' command.

Usage:
    repo abandon --all [<project>...]

Change-Id: I4d391f37fb9d89b8095488c585468eafc1a35f31
2016-10-17 02:29:42 +00:00
David Pursehouse
82f67987a3 Merge "sync: Fix semaphore release bug that causes thread 'leaks'" 2016-10-17 01:10:16 +00:00
Kyunam.jo
699bcd40be Removed duplication code in abandon.py
code about getting argument is duplicated.
so this line is removed

Change-Id: Id321b999c7dacdb403cd986cbf35f8db62efc157
2016-10-12 10:02:30 +09:00
Andrew Wheeler
7f1ccfbb7b sync: Fix semaphore release bug that causes thread 'leaks'
When repo syncs a manifest that utilizes multiple branches
in the same project, then the sync will use an extra
thread for each "duplicate".  For example, if
the manifest includes the project "foo" and "bar"
twice, then "repo sync -jN" will fetch with N+2 threads.

This is caused by _FetchHelper() releasing the thread semaphore
object each time it's called, even though _FetchProjectList()
may call this function multiple times within the scope of a
single thread.

Fix by moving the thread semaphore release to
_FetchProjectList(), which is only called once per thread
instance.

Change-Id: I1da78b145e09524d40457db5ca5c37d315432bd8
2016-10-11 14:10:34 -05:00
Dan Willemsen
eceeb1b1f5 Support broken symlinks when cleaning obsolete paths
When there's a symlink to a directory, os.walk still lists the symlink
in dirs, even if it isn't configured to follow symlinks. This will fail
the listdirs check if the symlink is broken (either before or during the
cleanup). So instead, check for directory symlinks and remove them using
os.remove.

Bug: Issue 231
Change-Id: I0ec45a26be566613a4a39bf694a3d9c6328481c2
2016-09-27 03:05:11 +00:00
Dan Willemsen
4350791e0d On project cleanup, don't remove nested projects
When there are nested projects in a manifest, like on AOSP right now:

<project path="build" name="platform/build" />
<project path="build/blueprint" name="platform/build/blueprint" />
<project path="build/kati" name="platform/build/kati" />
<project path="build/soong" name="platform/build/soong" />

And the top "build" project is removed (or renamed to remove the
nesting), repo just wipes away everything under build/ and re-creates
the projects that are still there. But it only checks to see if the
build/ project is dirty, so if there are dirty files in a nested
project, they'll just be blown away, and a fresh worktree checked out.

Instead, behave similarly to how `git clean -dxf` behaves and preserve
any subdirectories that have git repositories in them. This isn't as
strict as git -- it does not check to see if the '.git' entry is a
readable gitdir, just whether an entry named '.git' exists.

If it encounters any errors removing files, we'll print them all out to
stderr and tell the user that we were unable to clean up the obsolete
project, that they should clean it up manually, then sync again.

Change-Id: I2f6a7dd205a8e0b7590ca5369e9b0ba21d5a6f77
2016-09-20 17:16:12 -07:00
David Pursehouse
01b7d758d5 Merge "When syncing a project with a shared object store, disable automatic pruning." 2016-09-14 20:48:24 +00:00
David Pursehouse
d9de945d8a Merge "upload: short circuit when nothing is pending" 2016-09-14 05:20:36 +00:00
Gabe Black
2ff302929c When syncing a project with a shared object store, disable automatic pruning.
The shared object stores confuse git and make it throw away objects which are
still in use. We'll avoid that problem by disabling automatic pruning on those
projects, but there's nothing preventing a user from changing the config back
or pruning a repository manually.

BUG=chromium:375945
TEST=Ran repo sync on fresh ChromeOS checkout, starting with a branch of repo
with this change. Verified that the kernel projects and no others were
identified as having shared object stores, and that repo successfully disabled
automatic pruning in their configs. Re-enabled pruning and ran repo sync just
on one of the kernel directories. Verified that pruning was re-disabled as a
result.

Change-Id: I728ed5b06f0087aeb5a23ba8f5410a7cd10af5b0
2016-09-14 00:19:44 -04:00
Vadim Bendebury
e5c0ea0a95 Consider local project to be default for 'repo start'
The requirement to explicitly specify the local project when starting
a new repo branch is somewhat counter intuitive.

This patch uses the current directory's git tree as the default
project.

Tested by running

  'repo start <name>'

observed that the result is the same as if running

  'repo start <name> .'

Change-Id: If106caa801b4cd5ba70dbe8354a227d59f100aa3
2016-09-14 00:17:45 -04:00
Mike Frysinger
163a3be18b upload: short circuit when nothing is pending
When nothing is pending, most of this code is already short-circuited.
Hoist the single check up to make this more obvious/slightly faster.

Change-Id: Iec3a7e08eacd23a7c5f964900d5776bf5252c804
2016-09-14 00:16:37 -04:00
David Pursehouse
6a0a3648f1 Merge "init: Add --no-clone-bundle option" 2016-08-17 04:57:28 +00:00
David Pursehouse
6118faa118 Merge "init: Respect --quiet option when synching manifest repository" 2016-08-17 04:04:14 +00:00
Mike Frysinger
40252c20f7 RepoHook: allow users to approve hooks via manifests
The constant prompting when registered hooks change can be tedious and
has a large multiplication factor when the project is large (e.g. the
AOSP).  It gets worse as people want to write more checks, hooks, docs,
and tests (or fix bugs), but every CL that goes in will trigger a new
prompt to approve.

Let's tweak our trust model when it comes to hooks.  Since people start
off by calling `repo init` with a URL to a manifest, and that manifest
defines all the hooks, anchor trust in that.  This requires that we get
the manifest over a trusted link (e.g. https or ssh) so that it can't
be MITM-ed.  If the user chooses to use an untrusted link (e.g. git or
http), then we'll fallback to the existing hash based approval.

Bug: Issue 226
Change-Id: I77be9e4397383f264fcdaefb582e345ea4069a13
2016-08-16 13:02:52 -04:00