Commit Graph

1866 Commits

Author SHA1 Message Date
Mark E. Hamilton
4088eb434b repo: Cleaned up pylint/pep8 violations
I noticed when running pylint (as the SUBMITTING_PATCHES file directs)
that there were a number of violations reported. This makes it difficult
to see violations I might have introduced. This commit corrects all
pylint violations in the repo script.

First I ran this to clean up the formatting:

 autopep8 --max-line-length=80 --indent-size 2 repo

Following that the following violations remained:

% pylint --rcfile=.pylintrc repo
************* Module repo
W:220,21: Redefining name 'init_optparse' from outer scope (line 156)
(redefined-outer-name)
W:482, 2: No exception type(s) specified (bare-except)
C:704, 0: Old-style class defined. (old-style-class)

For line 220, the parameter to _GitcInitOptions was renamed so as not to
mask the init_optparse global.

For line 482, a pylint directive was added to disable the bare-execpt
violation for just that line.

For line 704, the _Options class was changed to subclass object.

Additionally, the comments at lines 107-113 were spaced out to line up
with the comment at line 112 that autopep8 moved.

This script now has a pylint score of 10.0

Change-Id: I779b66eb6b061a195d3c4372b99dec1b6d2a214f
2016-02-15 10:29:02 -07:00
Mark E. Hamilton
5553628601 repo: Add check of REPO_URL env variable
We want to be able to run repo on a system that is not connected to
the Internet and cannot access https://gerrit.googlesource.com. We
can put a clone of that repos there, but would prefer to use the
stable version of the repo script instead of a locally modified
version.

This commit adds a check for the REPO_URL environment variable. If
that is set and not empty its value will be set in the REPO_URL
global in repo.  Otherwise the standard path will be used.

Change-Id: I0616f5f81ef75f3463b73623b892cb5eed6bb7ba
2016-02-09 17:27:29 -07:00
Dan Willemsen
5ed805a98e Merge "Fix typos for manifest dtd" 2016-02-04 22:58:20 +00:00
Dan Willemsen
985ac6b946 Merge "Fix prune when bare git has detached head" 2016-02-04 22:44:20 +00:00
Dan Willemsen
ecf0a6c92b Merge "GITC: Fix 'repo start <branch> <repo>/<subdir>'" 2016-02-04 22:36:05 +00:00
Dan Willemsen
04197a5144 GITC: Fix 'repo start <branch> <repo>/<subdir>'
As soon as we wrote the gitc manifest, the folder for that repo became
empty, causing the next GetProjects lookup to fail. Reorder the
GetProjects calls so that they all happen while we still have the
repository contents available.

If you were already in a subdir, for cases like 'repo start <branch> .',
this would still fail, since the working directory would disappear out
from under you. That's fine most of the time, since we shouldn't be
doing operations based on the local directory, but git has a realpath
function that tries to restore CWD by chdir'ing back to it. So if the
working directory no longer exists, chdir to the topdir before
continuing.

Change-Id: Ibdf6cd37ff6e5a5f8338347c3919175491f7166f
2016-02-04 14:31:55 -08:00
Xiaohui Chen
0b4cb325c6 Add option to rebase onto project's manifest version
Some teams have a continuous build server that would mark certain
manifest green and safe to sync to.  Then team members could repo
sync to that particular manifest file and make sure they always
sync to a green build.  But if she/he has some local changes and
wants to rebase, currently it would be a manual process to find the
correct version to rebase onto.  This patch helps with that use
case by automating the process to rebase onto the currently synced
manifest version.

Change-Id: I847c9eb6addf7f84fd3f5594fbf8c0bcc103f9a5
2016-01-28 10:20:03 -08:00
Dan Willemsen
1a799d14b7 Fix prune when bare git has detached head
We don't really use HEAD much in the bare git repositories, but there
have been reports of errors in git-symbolic-ref:

  symbolic-ref: fatal: Refusing to point HEAD outside of refs/

That happen when the bare git repo is in the detached head state. It's
possible that previous operations were killed while we were pruning
branches.

Use DetachHead instead of SetHead if we're restoring the repo into a
detached head state.

Change-Id: I9062e8957bc70367d3ded399685ac026fbb421fc
2015-12-15 14:22:40 -08:00
Kenny Ho
827e547d9e Fix typos for manifest dtd
Change-Id: If53721544eca570e2bcce4598cdc2670a679c681
2015-12-11 00:05:54 +00:00
Hu Xiuyun
e9becc079c Sync: Fix error exit code when both -n and -f are used
When repo sync is used with -f (--force-error) and a project fails to
sync, the sync will continue but then exit with an error status.

However if -n (--network-only) is also used, the exit code is 0, even
when a project failed.

Modify the logic to make sure the sync exits with the correct status.

Bug: Issue 214
Change-Id: I0b5d97a34642c5aa3743750ef14a42c9d5743c1d
2015-11-26 02:25:43 +00:00
Dan Willemsen
466b8c4ea2 Set GIT_ALLOW_PROTOCOL to limit dangerous protocols
See git commit 33cfccbbf35a -- some protocols allow arbitrary command
execution as part of the URL. Instead of blindly allowing those,
whitelist the allowed URL protocols unless the user has already done so.

Bug: Issue 210
Change-Id: I6bd8e721aa5e3dab53ef28cfdc8fde33eb74ef76
2015-11-26 11:03:19 +09:00
Dan Willemsen
e1e0bd1f75 Check for broken links when updating linkfiles
If a linkfile is a broken link (destination does not exist), and it
needs to be updated, we didn't notice that it needed to be removed
first. Use lexists instead of exists to check for this condition.

Change-Id: I1f6a1f0193d3fd2b9f7a647836044997f6ab32eb
2015-11-18 16:51:51 -08:00
David Pursehouse
74cfd2709b Sync: Add option to prune refs during sync
By passing --prune to the sync command, the --prune option is
given to the `git fetch`, causing refs that no longer exist on
the remote to be removed.

Change-Id: I3cedacce14276d96ac2d5aabf2d07fd05e92bc02
2015-10-27 03:04:17 +00:00
Pascal Muetschard
c2a64ddffd A couple of fixes to the init command's -p option.
Adds windows as one of the allowed platforms flags.
Fixes -p foo to append 'platform-foo', instead of each letter (list.extend
expects a list and thus appends each char in the string, rather than the
string itself).

Change-Id: I73a92127ac29a32fc31b335cc54a246302904140
2015-10-22 13:28:20 -07:00
Dan Willemsen
745b4ad660 Fix gitc-init behavior
With gitc-init, a gitc client may be specified using '-c'. If we're
not currently in that client, we need to change directories so that
we don't affect the local checkout, and to ensure that repo is
checked out in the new client.

This also makes '-c' optional if already in a gitc client, to match
the rest of the init options.

Change-Id: Ib514ad9fd101698060ae89bb035499800897e9bd
2015-10-07 15:43:22 -07:00
David Pursehouse
4c5f74e452 Sync: Add HTTP Cookie File header on temporary cookie file
The .gitcookies file generated by googlesource.com does not have
the header:

 # (Netscape) HTTP Cookie File

which causes python's MozillaCookieJar.load to fail with the
error:

 "does not look like a Netscape format cookies file"

Prepend the expected header onto the generated cookie file.

We don't bother to check if the header already exists on the
file; repeating it does not cause any problem.

Bug: Issue 207
Change-Id: I7d39720a1d36a6aae00f70691156514ebc04e579
2015-10-02 11:12:05 +09:00
David Pursehouse
b1ad2190a2 Sync: Don't fail when git cookies can't be loaded
If the git cookies file fails to load, use a default
cookie jar instead.

Bug: Issue 207
Change-Id: I7cb326c204f2784ab4dbd13801b3186667af5b78
2015-10-02 11:04:01 +09:00
Simran Basi
f231db11a2 GITC: Add repo gitc-delete command.
repo gitc-delete deletes a GITC client and all the locally
saved sources. Useful for removing unnecessary clients and
recovering disk space.

Change-Id: Idf23addcea52b8713d268c34a7b37da0c5e5cd26
2015-10-01 21:05:17 +00:00
Dan Willemsen
79360640f4 Add GitcClientCommand class for GITC-specific commands
These won't show up as common commands in the help text unless in a GITC
client, and will refuse to execute.

Change-Id: Iffe82adcc9d6ddde9cb4b204f83ff018042bdab0
2015-09-29 13:46:34 -07:00
Dan Willemsen
7b01b2fd01 Merge "launcher: Update repo after applying clone.bundle" 2015-09-14 23:40:05 +00:00
David Pursehouse
aad84232ca Merge "docs: add copyfile and linkfile elements description" 2015-09-14 10:39:21 +00:00
Alexander Bird
3c03580607 fixed typo in gitc_init.py help output
Change-Id: I86459bf63297487457d6c4c995dfd1e63133ec53
2015-09-11 14:11:30 -04:00
Ruslan Bilovol
54527e7e30 docs: add copyfile and linkfile elements description
The "copyfile" element is available since 2009 and
have been used in every Android manifest; the "linkfile"
element is available since 2014.
Now it's a good time to add both to the documentation

Change-Id: Ia987edf5f69a006235fbd3f33b744e9794a6d964
Signed-off-by: Ruslan Bilovol <ruslan.bilovol@gmail.com>
2015-09-10 09:43:19 +00:00
Dan Willemsen
5ea32d1359 GITC: Always update the gitc manifest from the repo manifest
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.

This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.

Change-Id: I435235cb5622a048ffad0059affd32ecf71f1f5b
2015-09-09 20:50:40 -07:00
Dan Willemsen
5cc384034d Merge "Revert "GITC: Always update the gitc manifest from the repo manifest"" 2015-09-09 21:44:11 +00:00
Dan Willemsen
0375523331 Revert "GITC: Always update the gitc manifest from the repo manifest"
This reverts commit 250303b437.

Change-Id: I1fd8af20f802553151aacb953c913f3305ca6057
2015-09-09 21:43:32 +00:00
Dan Willemsen
c32ba1961e Merge "_CopyAndLinkFiles even if the sources haven't changed" 2015-09-09 20:47:19 +00:00
Dan Willemsen
250303b437 GITC: Always update the gitc manifest from the repo manifest
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.

This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.

Change-Id: I5dbc850dd73a9fbd10ab2470ae4c40e46ff894de
2015-09-09 12:35:56 -07:00
Dan Willemsen
029eaf3bac _CopyAndLinkFiles even if the sources haven't changed
The source or destination attributes may have changed even if the source
didn't, so we need to make sure that these are up to date.

Change-Id: I266ef3598ddda7e8c23bc9c6a049905ddc586348
2015-09-03 12:54:06 -07:00
Simran Basi
ba72d8301e GITC: Fix repo sync.
Fixing http://b/23785024 by calling os.getcwd() because variable
cwd no longer exists.

Change-Id: I21ff7d059e072f9f60726db76b67587a92c878ad
2015-09-03 10:47:44 -07:00
Dan Willemsen
fee390eea2 launcher: Update repo after applying clone.bundle
If the clone.bundle is out of date, repo may be installed with an old
version. It will upgrade with the next sync a day later, or when "repo
selfupdate" is run.

This behavior was added to normal project downloads, but was never added
to the repo launcher.

Change-Id: Ib04bef3a658c98fe1b6c53b3e8d0067165a5e3f7
2015-09-02 12:45:19 -07:00
Dan Willemsen
9ff2ece6ab gitc: Improve help visibility
This improves the visiblity of gitc-init if we can get the gitc config,
and hides it otherwise.

Change-Id: I82830b0b07c311e8c74397ba79eb4c361f8b6fb5
2015-09-01 12:23:56 -07:00
Dan Willemsen
2487cb7b2c Fix gitc check if gitc isn't installed
This was doing cwd.startswith(''), which is always true.

Change-Id: Icc059c09492b31e2d7651e4a595bda783c5abc47
2015-08-31 15:59:54 -07:00
Simran Basi
8ce5041596 GITC: Pull GITC Manifest Dir from the config.
Updates the repo launcher and gitc_utils to pull the manifest
directory location out of the gitc config file.

Change-Id: Id08381b8a7d61962093d5cddcb3ff6afbb13004b
2015-08-31 21:39:17 +00:00
Simran Basi
f7a51898d3 GITC: Expand relative remote URLs.
The GITC filesystem does not understand relative URLs for remotes,
so now if a remote uses a relative URL, it will be be expanded to
be relative to the manifest URL.

Change-Id: Ie1210758560aeb1934da3f71496aaf19c2728214
2015-08-28 18:09:05 +00:00
Simran Basi
b9a1b73425 GITC: Add repo start support.
Add repo start support for GITC checkouts. If the user is in
the GITC FS view, they can now run repo start to check out
the sources and create a new working branch.

When "repo start" is called on a GITC project, the revision
tag is set to an empty string and saved in a new tag:
old-revision. This tells the GITC filesystem to display the
local copy of the sources when being viewed. The local copy
is created by pulling the project sources and the new branch
is created based off the original project revision.

Updated main.py to setup each command's gitc_manifest when
appropriate.

Updated repo sync's logic to sync opened projects and
updating the GITC manifest file for the rest.

Change-Id: I7e4809d1c4fc43c69b26f2f1bebe45aab0cae628
2015-08-28 10:53:05 -07:00
David Pursehouse
dc2545cad6 project.py: Improve message shown when hook is not replaced
If a hook file has been modified locally, it will not be replaced.

Improve the message to make this clearer.

Also change it from an error to a warning.

Change-Id: I62c635390f24d2868db17717c247861b0381c99f
2015-08-25 05:40:46 +00:00
David Pursehouse
f33929d014 project.py: Consistently use the _error method to print error messages
Use the _error method instead of directly calling `print`.

Also add a new _warn convenience method.

Change-Id: Ia332c14ef8d9d1fe2df128dbf36b5521802ccdf1
2015-08-25 14:39:06 +09:00
Dan Willemsen
3010e5ba64 Smartsync: Don't fail if there isn't a cookiefile
Change-Id: I434a259f43ca9808e88051ac8ba865c519a24702
2015-08-20 10:29:37 -07:00
David Pursehouse
ba7bc738c1 Sync: Refactor netrc parsing
Don't emit a message when the netrc file doesn't exist or couldn't
be opened.

Instead of trying to unpack the result of info.authenticators() and
catching the resulting TypeError when it's None, first store it to
a local and only unpack it if it has a value.

Also remove an unused import.

Change-Id: I5c404d91e48c261c1ab850c3e5f040c4f4c235cb
2015-08-20 17:05:17 +09:00
David Pursehouse
f4599a2a3d gitc_init: Remove unused import
Also add missing newline at the end of the file.

Change-Id: I206e6c4b033d223eb0ff5824ecbf6fd98c39c918
2015-08-20 16:45:39 +09:00
David Pursehouse
022a1d4e6e gitc_utils: Fix incorrect string format argument
Change-Id: Ibbac6e111833c8f5d93cb6cb4a10f8f2c4fd8e11
2015-08-20 16:41:04 +09:00
David Pursehouse
41d1baac31 gitc_utils: Remove unused variable
Change-Id: I569819675a99ff6c01fb57b23fed033c39d14d1f
2015-08-20 16:40:44 +09:00
David Pursehouse
46496d8761 gitc_utils: Remove unused import; add missing import
shutils is not used. Remove it.

sys is used, but not imported.  Import it.

Change-Id: I4ee582f33bbd451601097ef2f20bee23b54764cd
2015-08-20 16:37:09 +09:00
Dan Willemsen
7c9263bce0 Merge "Support smart-sync through persistent-http[s]" 2015-08-19 23:36:35 +00:00
Dan Willemsen
dab9e99f0f Merge "Fix formatting of message when retrying clone" 2015-08-19 17:54:50 +00:00
Dan Willemsen
c5f15bf7c0 Merge "Include project path in --force-sync error message" 2015-08-19 17:53:28 +00:00
Dan Willemsen
6d35d676db Merge "Copy clone-depth in repo manifest" 2015-08-19 17:50:15 +00:00
Dan Willemsen
0745bb2657 Support smart-sync through persistent-http[s]
Use the same cookies and proxy that git traffic goes through for
persistent-http[s] to support authentication for smart-sync.

Change-Id: I20f4a281c259053a5a4fdbc48b1bca48e781c692
2015-08-19 10:22:11 -07:00
David Pursehouse
25857b8988 Fix formatting of message when retrying clone
Passing the force_sync variable into the string formatting results in
the message:

  "Retrying clone after deleting None"

or

  "Retrying clone after deleting True".

Pass the name of the git directory instead.

Also, move the print inside the if-block so it's only displayed
when the retry is actually going to be attempted.

Change-Id: I76d9ecc176cecee4ad512d13e9d1f6bd36aacbbb
2015-08-19 13:03:13 +00:00