Commit Graph

322 Commits

Author SHA1 Message Date
Hu Xiuyun
e9becc079c Sync: Fix error exit code when both -n and -f are used
When repo sync is used with -f (--force-error) and a project fails to
sync, the sync will continue but then exit with an error status.

However if -n (--network-only) is also used, the exit code is 0, even
when a project failed.

Modify the logic to make sure the sync exits with the correct status.

Bug: Issue 214
Change-Id: I0b5d97a34642c5aa3743750ef14a42c9d5743c1d
2015-11-26 02:25:43 +00:00
David Pursehouse
74cfd2709b Sync: Add option to prune refs during sync
By passing --prune to the sync command, the --prune option is
given to the `git fetch`, causing refs that no longer exist on
the remote to be removed.

Change-Id: I3cedacce14276d96ac2d5aabf2d07fd05e92bc02
2015-10-27 03:04:17 +00:00
Pascal Muetschard
c2a64ddffd A couple of fixes to the init command's -p option.
Adds windows as one of the allowed platforms flags.
Fixes -p foo to append 'platform-foo', instead of each letter (list.extend
expects a list and thus appends each char in the string, rather than the
string itself).

Change-Id: I73a92127ac29a32fc31b335cc54a246302904140
2015-10-22 13:28:20 -07:00
Dan Willemsen
745b4ad660 Fix gitc-init behavior
With gitc-init, a gitc client may be specified using '-c'. If we're
not currently in that client, we need to change directories so that
we don't affect the local checkout, and to ensure that repo is
checked out in the new client.

This also makes '-c' optional if already in a gitc client, to match
the rest of the init options.

Change-Id: Ib514ad9fd101698060ae89bb035499800897e9bd
2015-10-07 15:43:22 -07:00
David Pursehouse
4c5f74e452 Sync: Add HTTP Cookie File header on temporary cookie file
The .gitcookies file generated by googlesource.com does not have
the header:

 # (Netscape) HTTP Cookie File

which causes python's MozillaCookieJar.load to fail with the
error:

 "does not look like a Netscape format cookies file"

Prepend the expected header onto the generated cookie file.

We don't bother to check if the header already exists on the
file; repeating it does not cause any problem.

Bug: Issue 207
Change-Id: I7d39720a1d36a6aae00f70691156514ebc04e579
2015-10-02 11:12:05 +09:00
David Pursehouse
b1ad2190a2 Sync: Don't fail when git cookies can't be loaded
If the git cookies file fails to load, use a default
cookie jar instead.

Bug: Issue 207
Change-Id: I7cb326c204f2784ab4dbd13801b3186667af5b78
2015-10-02 11:04:01 +09:00
Simran Basi
f231db11a2 GITC: Add repo gitc-delete command.
repo gitc-delete deletes a GITC client and all the locally
saved sources. Useful for removing unnecessary clients and
recovering disk space.

Change-Id: Idf23addcea52b8713d268c34a7b37da0c5e5cd26
2015-10-01 21:05:17 +00:00
Dan Willemsen
79360640f4 Add GitcClientCommand class for GITC-specific commands
These won't show up as common commands in the help text unless in a GITC
client, and will refuse to execute.

Change-Id: Iffe82adcc9d6ddde9cb4b204f83ff018042bdab0
2015-09-29 13:46:34 -07:00
Alexander Bird
3c03580607 fixed typo in gitc_init.py help output
Change-Id: I86459bf63297487457d6c4c995dfd1e63133ec53
2015-09-11 14:11:30 -04:00
Dan Willemsen
5ea32d1359 GITC: Always update the gitc manifest from the repo manifest
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.

This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.

Change-Id: I435235cb5622a048ffad0059affd32ecf71f1f5b
2015-09-09 20:50:40 -07:00
Dan Willemsen
0375523331 Revert "GITC: Always update the gitc manifest from the repo manifest"
This reverts commit 250303b437.

Change-Id: I1fd8af20f802553151aacb953c913f3305ca6057
2015-09-09 21:43:32 +00:00
Dan Willemsen
250303b437 GITC: Always update the gitc manifest from the repo manifest
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.

This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.

Change-Id: I5dbc850dd73a9fbd10ab2470ae4c40e46ff894de
2015-09-09 12:35:56 -07:00
Simran Basi
ba72d8301e GITC: Fix repo sync.
Fixing http://b/23785024 by calling os.getcwd() because variable
cwd no longer exists.

Change-Id: I21ff7d059e072f9f60726db76b67587a92c878ad
2015-09-03 10:47:44 -07:00
Dan Willemsen
9ff2ece6ab gitc: Improve help visibility
This improves the visiblity of gitc-init if we can get the gitc config,
and hides it otherwise.

Change-Id: I82830b0b07c311e8c74397ba79eb4c361f8b6fb5
2015-09-01 12:23:56 -07:00
Simran Basi
8ce5041596 GITC: Pull GITC Manifest Dir from the config.
Updates the repo launcher and gitc_utils to pull the manifest
directory location out of the gitc config file.

Change-Id: Id08381b8a7d61962093d5cddcb3ff6afbb13004b
2015-08-31 21:39:17 +00:00
Simran Basi
f7a51898d3 GITC: Expand relative remote URLs.
The GITC filesystem does not understand relative URLs for remotes,
so now if a remote uses a relative URL, it will be be expanded to
be relative to the manifest URL.

Change-Id: Ie1210758560aeb1934da3f71496aaf19c2728214
2015-08-28 18:09:05 +00:00
Simran Basi
b9a1b73425 GITC: Add repo start support.
Add repo start support for GITC checkouts. If the user is in
the GITC FS view, they can now run repo start to check out
the sources and create a new working branch.

When "repo start" is called on a GITC project, the revision
tag is set to an empty string and saved in a new tag:
old-revision. This tells the GITC filesystem to display the
local copy of the sources when being viewed. The local copy
is created by pulling the project sources and the new branch
is created based off the original project revision.

Updated main.py to setup each command's gitc_manifest when
appropriate.

Updated repo sync's logic to sync opened projects and
updating the GITC manifest file for the rest.

Change-Id: I7e4809d1c4fc43c69b26f2f1bebe45aab0cae628
2015-08-28 10:53:05 -07:00
Dan Willemsen
3010e5ba64 Smartsync: Don't fail if there isn't a cookiefile
Change-Id: I434a259f43ca9808e88051ac8ba865c519a24702
2015-08-20 10:29:37 -07:00
David Pursehouse
ba7bc738c1 Sync: Refactor netrc parsing
Don't emit a message when the netrc file doesn't exist or couldn't
be opened.

Instead of trying to unpack the result of info.authenticators() and
catching the resulting TypeError when it's None, first store it to
a local and only unpack it if it has a value.

Also remove an unused import.

Change-Id: I5c404d91e48c261c1ab850c3e5f040c4f4c235cb
2015-08-20 17:05:17 +09:00
David Pursehouse
f4599a2a3d gitc_init: Remove unused import
Also add missing newline at the end of the file.

Change-Id: I206e6c4b033d223eb0ff5824ecbf6fd98c39c918
2015-08-20 16:45:39 +09:00
Dan Willemsen
7c9263bce0 Merge "Support smart-sync through persistent-http[s]" 2015-08-19 23:36:35 +00:00
Dan Willemsen
0745bb2657 Support smart-sync through persistent-http[s]
Use the same cookies and proxy that git traffic goes through for
persistent-http[s] to support authentication for smart-sync.

Change-Id: I20f4a281c259053a5a4fdbc48b1bca48e781c692
2015-08-19 10:22:11 -07:00
Simran Basi
bdb5271de3 GITC: Add repo sync support.
Add repo sync support for GITC checkouts. If the user is in the
GITC client directory they can still pull the sources as normal
if they pass in the --force-gitc argument. Otherwise the user
should call repo sync in the GITC view to update the user's
remote view. (This works because .repo in the GITC view will
link to .repo in the client config directory.)

Part of the support for this change is the refactoring of GITC
related code into gitc_utils.py.

Change-Id: I2636aaa50b450b6f091309db8dd0e8f4dbdad579
2015-08-18 11:59:10 -07:00
Dan Willemsen
5d0c3a614e Merge "GITC: Add gitc-init subcommand to repo." 2015-08-12 23:25:20 +00:00
Simran Basi
1efc2b4a01 GITC: Add gitc-init subcommand to repo.
Adds the new gitc-init command to set up a GITC client. Gitc-init
sets up the client directory and calls repo init within it. Once
the repo is initialized, then generates a GITC manifest file
by using git ls-remote on each project and retrieving the HEAD SHA
to use as the revision attribute.

Gitc-init inherits from and has all the options as repo init.

Change-Id: Icd7e47e90eab752a77de7c80ebc98cfe16bf6de3
2015-08-12 16:22:14 -07:00
David Pursehouse
43322283dc Merge "Support filtering by group on forall and list subcmd" 2015-08-05 01:01:02 +00:00
Conley Owens
555be54790 Merge "Emit project info in case of sync exception." 2015-07-31 17:06:23 +00:00
Dan Sandler
c5cd433daf Emit project info in case of sync exception.
Previously repo would only print the failing project path if
Sync_NetworkHalf returned false/empty, but if it threw an
exception the print() was never called.

Change-Id: I58c41de43930df5e34b21561c205e062a72e290f
2015-07-31 14:03:50 +00:00
Graham Christensen
0369a069ad Support filtering by group on forall and list subcmd
Enable operating against groups of repositories. As it stands, it isn't
compatible with `-r/--regex`.

`repo forall -g groupname -c pwd` will  run `pwd` for all projects in
groupname.

`repo forall -g thisgroup,-butnotthisone -c pwd` will  run `pwd` for all
projects in `thisgroup` but not `butnotthisone`.

`repo list -g groupname -n` will list all the names of repos in
`groupname`.

Change-Id: Ia75c50ce52541d1c8cea2874b20a4db2e0e54960
2015-07-30 12:59:35 -05:00
Kevin Degi
abaa7f312f Add option to correct gitdir when syncing
In some cases, a user may wish to continue with a sync even though
it would require overwriting an existing git directory. This behavior
is not safe as a default because it could result in the loss of some
user data, but as an optional flag it allows the user more flexibility.

To support this, add a --force-sync flag to the sync command that will
attempt to overwrite the existing git dir if it is specified and the
existing git dir points to the wrong obj dir.

Change-Id: Ieddda8ad54e264a1eb4a9d54881dd6ebc8a03833
2015-07-29 14:44:46 -06:00
David Pursehouse
57f43f4944 Merge "Prevent repo info from crashing when default element doesn't exist." 2015-07-29 02:11:55 +00:00
Cassidy Burden
17af578d72 Prevent repo info from crashing when default element doesn't exist.
repo info will crash when using a manifest with no default element despite
default being an optional element. Output nothing for "Manifest Branch" if no
default element exists (or if no default revision exists).

Change-Id: I7ebffa2408863837ba980f0ab6e593134400aea9
2015-07-27 16:56:31 -07:00
David Pursehouse
6944cdb8d1 forall: use smart sync override manifest if it exists
If a workspace is synced with the -s or -t option, the included projects
may be different to those in the original manifest. However, when using
the forall command, the list of the projects from the original manifest
is used.

If the smart sync manifest file exists, use it to override the original
manifest.

Change-Id: Iaefcbe148d2158ac046f158d98bbd8b5a5378ce7
2015-07-06 16:18:06 +09:00
David Pursehouse
59b417493e sync: Remove smart sync override manifest when not in smart sync mode
When syncing with the -s or -t option, a smart_sync_override.xml file
is created. This file is left in the file system when syncing again
without the -s or -t option.

Remove the smart sync override manifest, if it exists, when not using
the -s or -t option.

Change-Id: I697a0f6405205ba5f84a4d470becf7cd23c07b4b
2015-07-06 16:18:06 +09:00
David Pursehouse
30d13eea86 forall: Don't try to get lrev of projects in mirror workspace
git rev-parse fails for projects that don't have an explicit revision
specified, and don't have a branch of the same name as the default
revision. This can be the case in a workspace synced with the smart
sync (-s) or smart tag (-t) option.

Change-Id: I19bfe9fe7396170379415d85f10f6440dc6ea08f
2015-07-06 16:18:06 +09:00
David Pursehouse
727cc3e324 sync: Improve error message when writing smart sync manifest fails
The error message only states that writing the manifest failed.

Include the exception message, so it's easier to track down the reason
that the write failed.

Change-Id: I06e942c48a19521ba45292199519dd0a8bdb1de7
2015-07-06 16:18:06 +09:00
David Pursehouse
c5ceeb1625 Merge "Fix 'repo cherry-pick' to avoid hanging on commit-msg update." 2015-06-25 14:53:46 +00:00
Than McIntosh
db75704bfc Fix 'repo cherry-pick' to avoid hanging on commit-msg update.
After performing the actual cherry-pick operation, the code
in cherry_pick.py opens a pipe to 'git commit -F' to rewrite the commit
message, emits the fixed-up commit msg to the pipe, then waits
for 'git commit' to complete. The child 'git' process winds up
hanging while reading from the pipe, however, since the parent
process still has it open. To fix the hang, change the parent process
to close its end of the pipe after it has emitted the message.

Change-Id: I5929371e69a5b076f09009d00d40a2c72ac8ac33
2015-06-22 08:00:20 -04:00
Anthony King
c116f94261 forall: setenv, only encode val if encode exists
Change-Id: I655e3043d0118c4e929897d3a51e5e013e5758dc
2015-06-04 00:34:19 +00:00
Anthony King
7993f3cdda init: don't call urllib.parse
it's actually urllib.parse.urlparse

Change-Id: Ie3532e54625e887c8682d92b932ea21a629e8d60
2015-06-04 00:33:33 +00:00
Colin Cross
31a7be561e Catch exceptions in project list generator
If the generator that produces per-project worker arguments raises an
exception it triggers python bug http://bugs.python.org/issue8296.
Rewrite the generator expression as a generator function, and catch
Exceptions and KeyboardInterrupts to end the iteration.

Also add a pool worker initializer to disable SIGINT to prevent
KeyboardInterrupts inside multiprocessing.Pool in the worker threads
causing the same problem.

Fixes easy-to-reproduce hangs when hitting ctrl-c during
repo forall -c echo

Change-Id: Ie4a65b3e1e07a64ed6bb6ff20f3912c4326718ca
2015-05-13 11:09:38 -07:00
David Pursehouse
b155354034 Add option on sync to avoid fetching from remotes for existing sha1
In 2fb6466f79 an optimisation was
added to avoid fetching from remotes if the project is fixed to
a revision and the revision is already available locally.

This causes problems for users who expect all objects to be
fetched by default.

Change the logic so that the optimized behaviour is only enabled if
an option is explicitly given to repo sync.

Change-Id: I3b2794ddd8e0071b1787e166463cd8347ca9e24f
2015-04-30 14:29:02 +00:00
Conley Owens
a38769cda8 Merge "forall: use a generator to map the Pool" 2015-04-08 17:59:58 +00:00
Anthony King
6ad6dbefe7 forall: use a generator to map the Pool
Before, a list was generated, which is why there was a massive delay.

Using a generator will allow processes to start straight away

Change-Id: Ia325b0b340cc328c08c9bcc92a6709bbdaf6a664
2015-04-08 13:22:34 +01:00
Anthony King
b51f07cd06 status: lose dependence on StringIO
buflist was being used, which isn't available in Python 3.

`Execute` was using StringIO to capture the output of `PrintWorkTreeStatus`,
only to redirect it straight to stdout.
Instead, just let `PrintWorkTreeStatus` do it's own thing directly to stdout.

for handling `_FindOrphans`, we swap StringIO for a list. Nothing was done
that needed a a file like object.

Change-Id: Ibdaae137904de66a5ffb590d84203ef0fe782d8b
2015-04-04 21:21:49 +01:00
Jonathan Nieder
9371979628 Revert "Implementation of manifest defined githooks"
This reverts commit 38e4387f8e.

A "repo init" followed by "repo sync" is meant to be as safe as
"git clone".  In particular it should not run arbitrary code provided
by the manifest owner.

It would still be nice to have support for manifest-defined git hooks
--- they'd just need a prompt like the upload RepoHook has.  Hopefully
a later change can bring them back.

Change-Id: I5ecd90fb5c2ed64f103d856d1ffcba38a47b062d
Signed-off-by: Jonathan Nieder <jrn@google.com>
2015-03-17 11:29:58 -07:00
Conley Owens
85e8267031 Merge "Implementation of manifest defined githooks" 2015-03-05 20:52:30 +00:00
Jimmie Wester
38e4387f8e Implementation of manifest defined githooks
When working within a team or corporation it is often
useful/required to use predefined git templates. This
change teaches repo to use a per-remote git hook template
structure.

The implementation is done as a continuation of the
existing projecthook functionality. The terminology is
therefore defined as projecthooks.

The downloaded projecthooks are stored in the .repo
directory as a metaproject separating them from the users
project forest.

The projecthooks are downloaded and set up when doing a
repo init and updated for each new repo init.

When downloading a mirror the projecthooks gits are
not added to the bare forest since the intention is to
ensure that the latest are used (allows for company policy
enforcement).

The projecthooks are defined in the manifest file in the
remote element as a subnode, the name refers to the
project name on the server referred to in the remote.
<remote name="myremote ..>
   <projecthook name="myprojecthookgit" revision="myrevision"/>
</remote>

The hooks found in the projecthook revision supersede
the stock hooks found in repo. This removes the need for
updating the projecthook gits for repo stock hook changes.

Change-Id: I6796b7b0342c1f83c35f4b3e46782581b069a561
Signed-off-by: Patrik Ryd <patrik.ryd@stericsson.com>
Signed-off-by: Ian Kumlien <ian.kumlien@gmail.com>
2015-02-03 16:01:15 +09:00
David Pursehouse
f2fad61bde Add missing documentation of --current-branch option on sync command
Change-Id: I72d6e3d51241148c1df97bbad26338debb1fcb4e
2015-01-29 14:36:28 +09:00
Anthony King
7bdac71087 pylint fixes for project.py
Fix all the formatting warnings and unused variables

Change-Id: I17d88a23572303879530077f3a80451de5417fbb
2015-01-22 04:20:21 +00:00