Commit Graph

1635 Commits

Author SHA1 Message Date
Stefan Beller
d64e8eee51 pylint: ignore bad-whitespace
This ignores whitespaces errors, which we have quite a few of in argument
lists, for example:

************* Module git_config
C:209, 0: No space allowed around keyword argument assignment
  def HasSection(self, section, subsection = ''):
                                           ^ (bad-whitespace)
C:320, 0: No space allowed around keyword argument assignment
                   capture_stdout = True,
                                  ^ (bad-whitespace)
C:321, 0: No space allowed around keyword argument assignment
                   capture_stderr = True)
                                  ^ (bad-whitespace)
C:427, 0: Exactly one space required after comma
                     '-o','ControlPath %s' % ssh_sock(),
                         ^ (bad-whitespace)
C:436, 0: Exactly one space required after comma
    check_command = command_base + ['-O','check']
                                        ^ (bad-whitespace)
C:464, 0: Exactly one space required after comma
             % (host,port, str(e)), file=sys.stderr)
                    ^ (bad-whitespace)
C:707, 0: No space allowed around keyword argument assignment
    return self._config.GetString(key, all_keys = all_keys)
                                                ^ (bad-whitespace)
C:759, 0: No space allowed around keyword argument assignment
    return self._config.GetString(key, all_keys = all_keys)
                                                ^ (bad-whitespace)

Change-Id: Ia8f154f6741ce609787551f65877d7584c457903
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-06-17 16:37:24 -07:00
Dan Willemsen
8b39fb4bc0 Merge "Fix XmlManifest.Save with remotes that have 'alias' set" 2016-04-22 18:28:57 +00:00
Dan Willemsen
96c2d65489 Fix XmlManifest.Save with remotes that have 'alias' set
When the alias attribute is set for a remote, the RemoteSpec attached to
a Project only contains the alias name used by git, not the original
name used in the manifest. But that's not enough information to
reconstruct the manifest, so save off the original manifest name as
another RemoteSpec parameter, only used to write the manifest out.

Bug: Issue 181
Bug: Issue 219
Change-Id: Id7417dfd6ce5572e4e5fe14f22924fdf088ca4f3
2016-04-22 10:32:06 +09:00
Sebastian Schuberth
7ecccf6225 diffmanifests: support custom git pretty format strings
Change-Id: I29f4f1351c421f393328514d145df1a96aed9ee2
2016-04-21 18:36:11 +00:00
David Pursehouse
cee5c77166 Add a .mailmap file
Change-Id: I3c7e68fae0f8c082b2e0fbfc26cfb7dda31f1d34
2016-04-18 19:08:23 +09:00
David Pursehouse
79fba68e40 sync: Update help text for --smart-sync to be more specific
The --smart-sync option should return the manifest for *the latest*
known good build.

Change-Id: I2f3216b5b9e1af2ea5f9c3bf1c025813a3b77581
2016-04-13 18:03:00 +09:00
David Pursehouse
e868841782 Improve documentation of manifest server RPC methods
Mention that the RPC endpoints are used when running repo
sync with the --smart-sync and --smart-tag options

Change-Id: I4b0b82e8b714fe923a5b325a6135f0128bf636ff
2016-04-13 17:55:36 +09:00
Mark E. Hamilton
f9fe3e14d2 repo: Repo does not always handle '.' parameter correctly
The repo script allows a manifest to specify a '.' as the path the
top-level directory, which co-locates the .git and .repo directories,
and places files from the git repository at the top-level:

  <project name="proj_name" path="." />
  <project name="sierra.other.git" path="other" />

Most commands work correctly with this setup. Some commands, however,
fail to find the project. For instance, 'repo sync' works, and 'repo sync .'
works in a sub-project ('other' in this case) but 'repo sync .' in the
top-level directory fails with the error:

error: project . not found

There are two reasons for this:

1. The self.worktree attribute of the Project object is not normalized,
so with a '.' for path its value would be '/my/project/root/.'. This is
fine when used as a path, since it's the same path as '/my/project/root',
but when used in a string comparison it fails. This commit applies
os.path.normpath() to that value before storing it.

2. The _GetProjectByPath method in command.py was not checking the path
against manifest.topdir, so even once it was normalized the project was
not found. This commit adds a check against manifest.topdir if the
loop drops out without finding a project.

Change-Id: Ic84d053f1bbb5a357cad566805d5a326ae8246d2
2016-04-08 00:07:52 +00:00
Dan Willemsen
bdb866ea76 Fix symlinking of new projects
We weren't copying these lists, so the += was actually changing the
underlying lists.

When a new project was added to the manifest, we run _CheckDirReference
against the manifest project with share_refs=True, which added the
working_tree_* to the shareable_* lists. Then, when we load the new
manifest and create the new project, it uses the lists that already
contain the working_tree_* files, even though we passed
share_refs=False.

This happens reliably under the above conditions, but doesn't seem to
happen when syncing a fresh tree. So we've got a mixture of links that
may need to be cleaned up later. This patch will just stop it from
happening in the future.

Change-Id: Ib7935bfad78af1e494a75e55134ec829f13c2a41
2016-04-05 17:44:09 -07:00
David Pursehouse
e121ad558d Merge "Ignore clone.bundle on HTTP 501, i.e. Not Implemented" 2016-04-05 21:39:29 +00:00
Takeshi Kanemoto
1f0564406b Add --inverse-regex option to forall subcommand
Make it possible to exclude projects using regex/wildcard.

The syntax is similar to that of the -r option, e.g.:

  repo forall -i ^platform/ ^device/ -c 'echo $REPO_PROJECT'

Change-Id: Id250de5665152228c044c79337d3ac15b5696484
2016-04-05 07:28:27 +00:00
David Pursehouse
936d6185eb Merge changes from topic 'pylint-pep8-cleanup'
* changes:
  command.py: Cleaned up pylint/pep8 violations
  project.py: Cleaned up pylint/pep8 violations
2016-03-15 00:31:12 +00:00
Mike Bjorge
9322964d14 Update commit-msg hook to version from Gerrit 2.12.1
Change-Id: I31b74aba998f8e83f370a759218777f2557a8872
2016-03-14 10:08:33 +09:00
Mike Frysinger
4aa4b211c6 RepoHook: set __file__ when running the hook
A common design pattern is to use __file__ to find the location of the
active python module to assist in output or loading of related assets.
The current hook systems runs the pre-upload.py hook in a context w/out
that set leading to runtime errors:

$ repo upload --cbr .
ERROR: Traceback (most recent call last):
  File ".../repo/project.py", line 481, in _ExecuteHook
    self._script_fullpath, 'exec'), context)
  File ".../repohooks/pre-upload.py", line 32, in <module>
    path = os.path.dirname(os.path.realpath(__file__))
NameError: name '__file__' is not defined

Define this variable in this context so code can safely use it.

Change-Id: If6331312445fa61d9351b59f83abcc1c99ae6748
2016-03-05 21:52:31 +00:00
Mark E. Hamilton
8ccfa74d12 command.py: Cleaned up pylint/pep8 violations
I noticed when running pylint (as the SUBMITTING_PATCHES file directs)
that there were a few violations reported. This makes it difficult
to see violations I might have introduced. This commit corrects all
pylint violations in the command.py script.

This script now has a pylint score of 10.0.

Change-Id: Ibb35fa9af0e0b9b40e02ae043682b3af23286748
2016-03-02 09:05:45 -07:00
Mark E. Hamilton
30b0f4e022 project.py: Cleaned up pylint/pep8 violations
I noticed when running pylint (as the SUBMITTING_PATCHES file directs)
that there were a number of violations reported. This makes it difficult
to see violations I might have introduced. This commit corrects all
pylint violations in the project.py script.

This script now has a pylint score of 10.0, and no violations reported
by pep8.

Change-Id: I1462fd84f5b6b4c0dc893052671373e7ffd838f1
2016-03-02 09:05:40 -07:00
Jonathan Nieder
203153e7bb Add rpc: to default protocol whitelist
Change-Id: I57e1c3d93c0ce56da9c487df65eb3d258e0260e8
2016-02-26 18:53:54 -08:00
Alexandre Garnier
4cfb6d7167 Better error display on forall
It was only displaying 'Project list error: GitError()'
without any useful info about the project nor the error

Change-Id: Iad66cbaa03cad1053b5ae9ecc90d7772aa42ac13
2016-02-18 01:29:54 +00:00
Mark E. Hamilton
b29e61133e SUBMITTING_PATCHES: Expand instructions
This commit adds additional instructions on getting patches submitted,
based on my recent experience doing so.

Change-Id: I8e0d37d316214cc9a39383414773aad181f83f18
2016-02-17 02:45:08 +00:00
Mark E. Hamilton
4088eb434b repo: Cleaned up pylint/pep8 violations
I noticed when running pylint (as the SUBMITTING_PATCHES file directs)
that there were a number of violations reported. This makes it difficult
to see violations I might have introduced. This commit corrects all
pylint violations in the repo script.

First I ran this to clean up the formatting:

 autopep8 --max-line-length=80 --indent-size 2 repo

Following that the following violations remained:

% pylint --rcfile=.pylintrc repo
************* Module repo
W:220,21: Redefining name 'init_optparse' from outer scope (line 156)
(redefined-outer-name)
W:482, 2: No exception type(s) specified (bare-except)
C:704, 0: Old-style class defined. (old-style-class)

For line 220, the parameter to _GitcInitOptions was renamed so as not to
mask the init_optparse global.

For line 482, a pylint directive was added to disable the bare-execpt
violation for just that line.

For line 704, the _Options class was changed to subclass object.

Additionally, the comments at lines 107-113 were spaced out to line up
with the comment at line 112 that autopep8 moved.

This script now has a pylint score of 10.0

Change-Id: I779b66eb6b061a195d3c4372b99dec1b6d2a214f
2016-02-15 10:29:02 -07:00
Mark E. Hamilton
5553628601 repo: Add check of REPO_URL env variable
We want to be able to run repo on a system that is not connected to
the Internet and cannot access https://gerrit.googlesource.com. We
can put a clone of that repos there, but would prefer to use the
stable version of the repo script instead of a locally modified
version.

This commit adds a check for the REPO_URL environment variable. If
that is set and not empty its value will be set in the REPO_URL
global in repo.  Otherwise the standard path will be used.

Change-Id: I0616f5f81ef75f3463b73623b892cb5eed6bb7ba
2016-02-09 17:27:29 -07:00
Dan Willemsen
5ed805a98e Merge "Fix typos for manifest dtd" 2016-02-04 22:58:20 +00:00
Dan Willemsen
985ac6b946 Merge "Fix prune when bare git has detached head" 2016-02-04 22:44:20 +00:00
Dan Willemsen
ecf0a6c92b Merge "GITC: Fix 'repo start <branch> <repo>/<subdir>'" 2016-02-04 22:36:05 +00:00
Dan Willemsen
04197a5144 GITC: Fix 'repo start <branch> <repo>/<subdir>'
As soon as we wrote the gitc manifest, the folder for that repo became
empty, causing the next GetProjects lookup to fail. Reorder the
GetProjects calls so that they all happen while we still have the
repository contents available.

If you were already in a subdir, for cases like 'repo start <branch> .',
this would still fail, since the working directory would disappear out
from under you. That's fine most of the time, since we shouldn't be
doing operations based on the local directory, but git has a realpath
function that tries to restore CWD by chdir'ing back to it. So if the
working directory no longer exists, chdir to the topdir before
continuing.

Change-Id: Ibdf6cd37ff6e5a5f8338347c3919175491f7166f
2016-02-04 14:31:55 -08:00
Xiaohui Chen
0b4cb325c6 Add option to rebase onto project's manifest version
Some teams have a continuous build server that would mark certain
manifest green and safe to sync to.  Then team members could repo
sync to that particular manifest file and make sure they always
sync to a green build.  But if she/he has some local changes and
wants to rebase, currently it would be a manual process to find the
correct version to rebase onto.  This patch helps with that use
case by automating the process to rebase onto the currently synced
manifest version.

Change-Id: I847c9eb6addf7f84fd3f5594fbf8c0bcc103f9a5
2016-01-28 10:20:03 -08:00
Dan Willemsen
1a799d14b7 Fix prune when bare git has detached head
We don't really use HEAD much in the bare git repositories, but there
have been reports of errors in git-symbolic-ref:

  symbolic-ref: fatal: Refusing to point HEAD outside of refs/

That happen when the bare git repo is in the detached head state. It's
possible that previous operations were killed while we were pruning
branches.

Use DetachHead instead of SetHead if we're restoring the repo into a
detached head state.

Change-Id: I9062e8957bc70367d3ded399685ac026fbb421fc
2015-12-15 14:22:40 -08:00
Kenny Ho
827e547d9e Fix typos for manifest dtd
Change-Id: If53721544eca570e2bcce4598cdc2670a679c681
2015-12-11 00:05:54 +00:00
Hu Xiuyun
e9becc079c Sync: Fix error exit code when both -n and -f are used
When repo sync is used with -f (--force-error) and a project fails to
sync, the sync will continue but then exit with an error status.

However if -n (--network-only) is also used, the exit code is 0, even
when a project failed.

Modify the logic to make sure the sync exits with the correct status.

Bug: Issue 214
Change-Id: I0b5d97a34642c5aa3743750ef14a42c9d5743c1d
2015-11-26 02:25:43 +00:00
Dan Willemsen
466b8c4ea2 Set GIT_ALLOW_PROTOCOL to limit dangerous protocols
See git commit 33cfccbbf35a -- some protocols allow arbitrary command
execution as part of the URL. Instead of blindly allowing those,
whitelist the allowed URL protocols unless the user has already done so.

Bug: Issue 210
Change-Id: I6bd8e721aa5e3dab53ef28cfdc8fde33eb74ef76
2015-11-26 11:03:19 +09:00
Dan Willemsen
e1e0bd1f75 Check for broken links when updating linkfiles
If a linkfile is a broken link (destination does not exist), and it
needs to be updated, we didn't notice that it needed to be removed
first. Use lexists instead of exists to check for this condition.

Change-Id: I1f6a1f0193d3fd2b9f7a647836044997f6ab32eb
2015-11-18 16:51:51 -08:00
David Pursehouse
74cfd2709b Sync: Add option to prune refs during sync
By passing --prune to the sync command, the --prune option is
given to the `git fetch`, causing refs that no longer exist on
the remote to be removed.

Change-Id: I3cedacce14276d96ac2d5aabf2d07fd05e92bc02
2015-10-27 03:04:17 +00:00
Pascal Muetschard
c2a64ddffd A couple of fixes to the init command's -p option.
Adds windows as one of the allowed platforms flags.
Fixes -p foo to append 'platform-foo', instead of each letter (list.extend
expects a list and thus appends each char in the string, rather than the
string itself).

Change-Id: I73a92127ac29a32fc31b335cc54a246302904140
2015-10-22 13:28:20 -07:00
Dan Willemsen
745b4ad660 Fix gitc-init behavior
With gitc-init, a gitc client may be specified using '-c'. If we're
not currently in that client, we need to change directories so that
we don't affect the local checkout, and to ensure that repo is
checked out in the new client.

This also makes '-c' optional if already in a gitc client, to match
the rest of the init options.

Change-Id: Ib514ad9fd101698060ae89bb035499800897e9bd
2015-10-07 15:43:22 -07:00
David Pursehouse
4c5f74e452 Sync: Add HTTP Cookie File header on temporary cookie file
The .gitcookies file generated by googlesource.com does not have
the header:

 # (Netscape) HTTP Cookie File

which causes python's MozillaCookieJar.load to fail with the
error:

 "does not look like a Netscape format cookies file"

Prepend the expected header onto the generated cookie file.

We don't bother to check if the header already exists on the
file; repeating it does not cause any problem.

Bug: Issue 207
Change-Id: I7d39720a1d36a6aae00f70691156514ebc04e579
2015-10-02 11:12:05 +09:00
David Pursehouse
b1ad2190a2 Sync: Don't fail when git cookies can't be loaded
If the git cookies file fails to load, use a default
cookie jar instead.

Bug: Issue 207
Change-Id: I7cb326c204f2784ab4dbd13801b3186667af5b78
2015-10-02 11:04:01 +09:00
Simran Basi
f231db11a2 GITC: Add repo gitc-delete command.
repo gitc-delete deletes a GITC client and all the locally
saved sources. Useful for removing unnecessary clients and
recovering disk space.

Change-Id: Idf23addcea52b8713d268c34a7b37da0c5e5cd26
2015-10-01 21:05:17 +00:00
Dan Willemsen
79360640f4 Add GitcClientCommand class for GITC-specific commands
These won't show up as common commands in the help text unless in a GITC
client, and will refuse to execute.

Change-Id: Iffe82adcc9d6ddde9cb4b204f83ff018042bdab0
2015-09-29 13:46:34 -07:00
Dan Willemsen
7b01b2fd01 Merge "launcher: Update repo after applying clone.bundle" 2015-09-14 23:40:05 +00:00
David Pursehouse
aad84232ca Merge "docs: add copyfile and linkfile elements description" 2015-09-14 10:39:21 +00:00
Alexander Bird
3c03580607 fixed typo in gitc_init.py help output
Change-Id: I86459bf63297487457d6c4c995dfd1e63133ec53
2015-09-11 14:11:30 -04:00
Ruslan Bilovol
54527e7e30 docs: add copyfile and linkfile elements description
The "copyfile" element is available since 2009 and
have been used in every Android manifest; the "linkfile"
element is available since 2014.
Now it's a good time to add both to the documentation

Change-Id: Ia987edf5f69a006235fbd3f33b744e9794a6d964
Signed-off-by: Ruslan Bilovol <ruslan.bilovol@gmail.com>
2015-09-10 09:43:19 +00:00
Dan Willemsen
5ea32d1359 GITC: Always update the gitc manifest from the repo manifest
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.

This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.

Change-Id: I435235cb5622a048ffad0059affd32ecf71f1f5b
2015-09-09 20:50:40 -07:00
Dan Willemsen
5cc384034d Merge "Revert "GITC: Always update the gitc manifest from the repo manifest"" 2015-09-09 21:44:11 +00:00
Dan Willemsen
0375523331 Revert "GITC: Always update the gitc manifest from the repo manifest"
This reverts commit 250303b437.

Change-Id: I1fd8af20f802553151aacb953c913f3305ca6057
2015-09-09 21:43:32 +00:00
Dan Willemsen
c32ba1961e Merge "_CopyAndLinkFiles even if the sources haven't changed" 2015-09-09 20:47:19 +00:00
Dan Willemsen
250303b437 GITC: Always update the gitc manifest from the repo manifest
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.

This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.

Change-Id: I5dbc850dd73a9fbd10ab2470ae4c40e46ff894de
2015-09-09 12:35:56 -07:00
Dan Willemsen
029eaf3bac _CopyAndLinkFiles even if the sources haven't changed
The source or destination attributes may have changed even if the source
didn't, so we need to make sure that these are up to date.

Change-Id: I266ef3598ddda7e8c23bc9c6a049905ddc586348
2015-09-03 12:54:06 -07:00
Simran Basi
ba72d8301e GITC: Fix repo sync.
Fixing http://b/23785024 by calling os.getcwd() because variable
cwd no longer exists.

Change-Id: I21ff7d059e072f9f60726db76b67587a92c878ad
2015-09-03 10:47:44 -07:00
Dan Willemsen
fee390eea2 launcher: Update repo after applying clone.bundle
If the clone.bundle is out of date, repo may be installed with an old
version. It will upgrade with the next sync a day later, or when "repo
selfupdate" is run.

This behavior was added to normal project downloads, but was never added
to the repo launcher.

Change-Id: Ib04bef3a658c98fe1b6c53b3e8d0067165a5e3f7
2015-09-02 12:45:19 -07:00