Commit Graph

1496 Commits

Author SHA1 Message Date
David Pursehouse
8016f60a46 Merge "repo: Repo does not always handle '.' parameter correctly" 2016-08-14 08:50:28 +00:00
Dāvis Mosāns
631d0ec708 Support non-ASCII GNUPGHOME environment variable
Here we don't need to encode this gpg_dir string when using
Python 2.7 on Linux.

Change-Id: I56724e9511d3b1aea61535e654a45c212130630d
2016-07-16 22:10:06 +03:00
Xin Li
f97e72e5dd Bail out when manifest is referencing a bad SHA-1 revision.
BUG: Issue 222
Change-Id: Ie0a64b39922d6fdf1be2989eb514985be8490278
2016-06-29 11:01:43 -07:00
Cheuk Leung
8ac0c96537 Fix removing broken symlink in reference dir
Re-ordered to first create the symlink before checking the source
file and remove the destination if the source does not exists.

Change-Id: Iae923ba2ef0ba5a8dc1b8e42d8cc3f3708f773af
2016-06-29 05:43:01 +00:00
David Pursehouse
faaddc9b4e Merge "Adds additional crlf clobber avoidance." 2016-06-29 05:42:20 +00:00
David Pursehouse
a36af0767b Merge "Fix variable assignment" 2016-06-29 01:43:41 +00:00
Aymen Bouaziz
037040f73e Fix variable assignment
If upstream string is empty, current_branch_only variable will be assigned
to an empty string.

This is not what we expect here as this variable is a boolean.

Change-Id: Ibba935e25a74c2be1e50c88b4b403cf394ba365e
2016-06-28 12:31:25 +02:00
Aymen Bouaziz
2598ed06f1 Fix submodule checkout error when using sync-s option
When sync-s="true" option is used, the checkout of a submodule will try
to use the revision attribute of the parent project.

If this revision is a named reference, the checkout will fail if there
is no reference with this name in the submodule.

The proposed solution is to use the git commit id as revisionExpr for
submodules.

Change-Id: Ie8390a11957fd6a9c61289c6861d13cb3fa11678
2016-06-27 20:16:45 +02:00
Fredrik Berggren
01952e6634 Adds additional crlf clobber avoidance.
Adds the hook-scripts to .gitattributes due to the shell-scripts not
liking CRLF which they will get if a user sets 'autocrlf = true'
in their global gitconfig.

Further, since the python interpreter can handle either CRLF or LF, 
python-scripts specific line-ending rules have been removed.

Change-Id: I2d6bfd491b2f626b9ca93c40a3a7f2cfba6c54f0
2016-06-22 08:36:45 +00:00
Stefan Beller
9d2b14d2ec pylint: Fix unused-{argument,variable} warning
This commit fixes 4 out of the remaining 5 pylint warnings:

$ pylint --rcfile=.pylintrc *.py
************* Module gitc_utils
W:146, 0: TODO(sbasi/jorg): Come up with a solution to remove the sleep below. (fixme)
W:130, 6: Unused variable 'name' (unused-variable)
************* Module main
W:382,32: Unused argument 'fp' (unused-argument)
W:382,36: Unused argument 'code' (unused-argument)
W:382,42: Unused argument 'msg' (unused-argument)

Change-Id: Ie3d77b9a65b7daefaa9aa4b80f4b682c1678fd58
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-06-21 11:48:57 -07:00
Stefan Beller
6685106306 pylint: fix indentation in manifest_xml
This fixes pylint warning:

************* Module manifest_xml
W:975, 0: Bad indentation. Found 8 spaces, expected 6 (bad-indentation)

Change-Id: I967212f9439430351836ebdc27e442d7b77476e2
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-06-17 16:45:48 -07:00
Stefan Beller
d64e8eee51 pylint: ignore bad-whitespace
This ignores whitespaces errors, which we have quite a few of in argument
lists, for example:

************* Module git_config
C:209, 0: No space allowed around keyword argument assignment
  def HasSection(self, section, subsection = ''):
                                           ^ (bad-whitespace)
C:320, 0: No space allowed around keyword argument assignment
                   capture_stdout = True,
                                  ^ (bad-whitespace)
C:321, 0: No space allowed around keyword argument assignment
                   capture_stderr = True)
                                  ^ (bad-whitespace)
C:427, 0: Exactly one space required after comma
                     '-o','ControlPath %s' % ssh_sock(),
                         ^ (bad-whitespace)
C:436, 0: Exactly one space required after comma
    check_command = command_base + ['-O','check']
                                        ^ (bad-whitespace)
C:464, 0: Exactly one space required after comma
             % (host,port, str(e)), file=sys.stderr)
                    ^ (bad-whitespace)
C:707, 0: No space allowed around keyword argument assignment
    return self._config.GetString(key, all_keys = all_keys)
                                                ^ (bad-whitespace)
C:759, 0: No space allowed around keyword argument assignment
    return self._config.GetString(key, all_keys = all_keys)
                                                ^ (bad-whitespace)

Change-Id: Ia8f154f6741ce609787551f65877d7584c457903
Signed-off-by: Stefan Beller <sbeller@google.com>
2016-06-17 16:37:24 -07:00
Dan Willemsen
8b39fb4bc0 Merge "Fix XmlManifest.Save with remotes that have 'alias' set" 2016-04-22 18:28:57 +00:00
Dan Willemsen
96c2d65489 Fix XmlManifest.Save with remotes that have 'alias' set
When the alias attribute is set for a remote, the RemoteSpec attached to
a Project only contains the alias name used by git, not the original
name used in the manifest. But that's not enough information to
reconstruct the manifest, so save off the original manifest name as
another RemoteSpec parameter, only used to write the manifest out.

Bug: Issue 181
Bug: Issue 219
Change-Id: Id7417dfd6ce5572e4e5fe14f22924fdf088ca4f3
2016-04-22 10:32:06 +09:00
Sebastian Schuberth
7ecccf6225 diffmanifests: support custom git pretty format strings
Change-Id: I29f4f1351c421f393328514d145df1a96aed9ee2
2016-04-21 18:36:11 +00:00
David Pursehouse
cee5c77166 Add a .mailmap file
Change-Id: I3c7e68fae0f8c082b2e0fbfc26cfb7dda31f1d34
2016-04-18 19:08:23 +09:00
David Pursehouse
79fba68e40 sync: Update help text for --smart-sync to be more specific
The --smart-sync option should return the manifest for *the latest*
known good build.

Change-Id: I2f3216b5b9e1af2ea5f9c3bf1c025813a3b77581
2016-04-13 18:03:00 +09:00
David Pursehouse
e868841782 Improve documentation of manifest server RPC methods
Mention that the RPC endpoints are used when running repo
sync with the --smart-sync and --smart-tag options

Change-Id: I4b0b82e8b714fe923a5b325a6135f0128bf636ff
2016-04-13 17:55:36 +09:00
Mark E. Hamilton
f9fe3e14d2 repo: Repo does not always handle '.' parameter correctly
The repo script allows a manifest to specify a '.' as the path the
top-level directory, which co-locates the .git and .repo directories,
and places files from the git repository at the top-level:

  <project name="proj_name" path="." />
  <project name="sierra.other.git" path="other" />

Most commands work correctly with this setup. Some commands, however,
fail to find the project. For instance, 'repo sync' works, and 'repo sync .'
works in a sub-project ('other' in this case) but 'repo sync .' in the
top-level directory fails with the error:

error: project . not found

There are two reasons for this:

1. The self.worktree attribute of the Project object is not normalized,
so with a '.' for path its value would be '/my/project/root/.'. This is
fine when used as a path, since it's the same path as '/my/project/root',
but when used in a string comparison it fails. This commit applies
os.path.normpath() to that value before storing it.

2. The _GetProjectByPath method in command.py was not checking the path
against manifest.topdir, so even once it was normalized the project was
not found. This commit adds a check against manifest.topdir if the
loop drops out without finding a project.

Change-Id: Ic84d053f1bbb5a357cad566805d5a326ae8246d2
2016-04-08 00:07:52 +00:00
Dan Willemsen
bdb866ea76 Fix symlinking of new projects
We weren't copying these lists, so the += was actually changing the
underlying lists.

When a new project was added to the manifest, we run _CheckDirReference
against the manifest project with share_refs=True, which added the
working_tree_* to the shareable_* lists. Then, when we load the new
manifest and create the new project, it uses the lists that already
contain the working_tree_* files, even though we passed
share_refs=False.

This happens reliably under the above conditions, but doesn't seem to
happen when syncing a fresh tree. So we've got a mixture of links that
may need to be cleaned up later. This patch will just stop it from
happening in the future.

Change-Id: Ib7935bfad78af1e494a75e55134ec829f13c2a41
2016-04-05 17:44:09 -07:00
David Pursehouse
e121ad558d Merge "Ignore clone.bundle on HTTP 501, i.e. Not Implemented" 2016-04-05 21:39:29 +00:00
Takeshi Kanemoto
1f0564406b Add --inverse-regex option to forall subcommand
Make it possible to exclude projects using regex/wildcard.

The syntax is similar to that of the -r option, e.g.:

  repo forall -i ^platform/ ^device/ -c 'echo $REPO_PROJECT'

Change-Id: Id250de5665152228c044c79337d3ac15b5696484
2016-04-05 07:28:27 +00:00
David Pursehouse
936d6185eb Merge changes from topic 'pylint-pep8-cleanup'
* changes:
  command.py: Cleaned up pylint/pep8 violations
  project.py: Cleaned up pylint/pep8 violations
2016-03-15 00:31:12 +00:00
Mike Bjorge
9322964d14 Update commit-msg hook to version from Gerrit 2.12.1
Change-Id: I31b74aba998f8e83f370a759218777f2557a8872
2016-03-14 10:08:33 +09:00
Mike Frysinger
4aa4b211c6 RepoHook: set __file__ when running the hook
A common design pattern is to use __file__ to find the location of the
active python module to assist in output or loading of related assets.
The current hook systems runs the pre-upload.py hook in a context w/out
that set leading to runtime errors:

$ repo upload --cbr .
ERROR: Traceback (most recent call last):
  File ".../repo/project.py", line 481, in _ExecuteHook
    self._script_fullpath, 'exec'), context)
  File ".../repohooks/pre-upload.py", line 32, in <module>
    path = os.path.dirname(os.path.realpath(__file__))
NameError: name '__file__' is not defined

Define this variable in this context so code can safely use it.

Change-Id: If6331312445fa61d9351b59f83abcc1c99ae6748
2016-03-05 21:52:31 +00:00
Mark E. Hamilton
8ccfa74d12 command.py: Cleaned up pylint/pep8 violations
I noticed when running pylint (as the SUBMITTING_PATCHES file directs)
that there were a few violations reported. This makes it difficult
to see violations I might have introduced. This commit corrects all
pylint violations in the command.py script.

This script now has a pylint score of 10.0.

Change-Id: Ibb35fa9af0e0b9b40e02ae043682b3af23286748
2016-03-02 09:05:45 -07:00
Mark E. Hamilton
30b0f4e022 project.py: Cleaned up pylint/pep8 violations
I noticed when running pylint (as the SUBMITTING_PATCHES file directs)
that there were a number of violations reported. This makes it difficult
to see violations I might have introduced. This commit corrects all
pylint violations in the project.py script.

This script now has a pylint score of 10.0, and no violations reported
by pep8.

Change-Id: I1462fd84f5b6b4c0dc893052671373e7ffd838f1
2016-03-02 09:05:40 -07:00
Jonathan Nieder
203153e7bb Add rpc: to default protocol whitelist
Change-Id: I57e1c3d93c0ce56da9c487df65eb3d258e0260e8
2016-02-26 18:53:54 -08:00
Alexandre Garnier
4cfb6d7167 Better error display on forall
It was only displaying 'Project list error: GitError()'
without any useful info about the project nor the error

Change-Id: Iad66cbaa03cad1053b5ae9ecc90d7772aa42ac13
2016-02-18 01:29:54 +00:00
Mark E. Hamilton
b29e61133e SUBMITTING_PATCHES: Expand instructions
This commit adds additional instructions on getting patches submitted,
based on my recent experience doing so.

Change-Id: I8e0d37d316214cc9a39383414773aad181f83f18
2016-02-17 02:45:08 +00:00
Mark E. Hamilton
4088eb434b repo: Cleaned up pylint/pep8 violations
I noticed when running pylint (as the SUBMITTING_PATCHES file directs)
that there were a number of violations reported. This makes it difficult
to see violations I might have introduced. This commit corrects all
pylint violations in the repo script.

First I ran this to clean up the formatting:

 autopep8 --max-line-length=80 --indent-size 2 repo

Following that the following violations remained:

% pylint --rcfile=.pylintrc repo
************* Module repo
W:220,21: Redefining name 'init_optparse' from outer scope (line 156)
(redefined-outer-name)
W:482, 2: No exception type(s) specified (bare-except)
C:704, 0: Old-style class defined. (old-style-class)

For line 220, the parameter to _GitcInitOptions was renamed so as not to
mask the init_optparse global.

For line 482, a pylint directive was added to disable the bare-execpt
violation for just that line.

For line 704, the _Options class was changed to subclass object.

Additionally, the comments at lines 107-113 were spaced out to line up
with the comment at line 112 that autopep8 moved.

This script now has a pylint score of 10.0

Change-Id: I779b66eb6b061a195d3c4372b99dec1b6d2a214f
2016-02-15 10:29:02 -07:00
Mark E. Hamilton
5553628601 repo: Add check of REPO_URL env variable
We want to be able to run repo on a system that is not connected to
the Internet and cannot access https://gerrit.googlesource.com. We
can put a clone of that repos there, but would prefer to use the
stable version of the repo script instead of a locally modified
version.

This commit adds a check for the REPO_URL environment variable. If
that is set and not empty its value will be set in the REPO_URL
global in repo.  Otherwise the standard path will be used.

Change-Id: I0616f5f81ef75f3463b73623b892cb5eed6bb7ba
2016-02-09 17:27:29 -07:00
Dan Willemsen
5ed805a98e Merge "Fix typos for manifest dtd" 2016-02-04 22:58:20 +00:00
Dan Willemsen
985ac6b946 Merge "Fix prune when bare git has detached head" 2016-02-04 22:44:20 +00:00
Dan Willemsen
ecf0a6c92b Merge "GITC: Fix 'repo start <branch> <repo>/<subdir>'" 2016-02-04 22:36:05 +00:00
Dan Willemsen
04197a5144 GITC: Fix 'repo start <branch> <repo>/<subdir>'
As soon as we wrote the gitc manifest, the folder for that repo became
empty, causing the next GetProjects lookup to fail. Reorder the
GetProjects calls so that they all happen while we still have the
repository contents available.

If you were already in a subdir, for cases like 'repo start <branch> .',
this would still fail, since the working directory would disappear out
from under you. That's fine most of the time, since we shouldn't be
doing operations based on the local directory, but git has a realpath
function that tries to restore CWD by chdir'ing back to it. So if the
working directory no longer exists, chdir to the topdir before
continuing.

Change-Id: Ibdf6cd37ff6e5a5f8338347c3919175491f7166f
2016-02-04 14:31:55 -08:00
Xiaohui Chen
0b4cb325c6 Add option to rebase onto project's manifest version
Some teams have a continuous build server that would mark certain
manifest green and safe to sync to.  Then team members could repo
sync to that particular manifest file and make sure they always
sync to a green build.  But if she/he has some local changes and
wants to rebase, currently it would be a manual process to find the
correct version to rebase onto.  This patch helps with that use
case by automating the process to rebase onto the currently synced
manifest version.

Change-Id: I847c9eb6addf7f84fd3f5594fbf8c0bcc103f9a5
2016-01-28 10:20:03 -08:00
Dan Willemsen
1a799d14b7 Fix prune when bare git has detached head
We don't really use HEAD much in the bare git repositories, but there
have been reports of errors in git-symbolic-ref:

  symbolic-ref: fatal: Refusing to point HEAD outside of refs/

That happen when the bare git repo is in the detached head state. It's
possible that previous operations were killed while we were pruning
branches.

Use DetachHead instead of SetHead if we're restoring the repo into a
detached head state.

Change-Id: I9062e8957bc70367d3ded399685ac026fbb421fc
2015-12-15 14:22:40 -08:00
Kenny Ho
827e547d9e Fix typos for manifest dtd
Change-Id: If53721544eca570e2bcce4598cdc2670a679c681
2015-12-11 00:05:54 +00:00
Hu Xiuyun
e9becc079c Sync: Fix error exit code when both -n and -f are used
When repo sync is used with -f (--force-error) and a project fails to
sync, the sync will continue but then exit with an error status.

However if -n (--network-only) is also used, the exit code is 0, even
when a project failed.

Modify the logic to make sure the sync exits with the correct status.

Bug: Issue 214
Change-Id: I0b5d97a34642c5aa3743750ef14a42c9d5743c1d
2015-11-26 02:25:43 +00:00
Dan Willemsen
466b8c4ea2 Set GIT_ALLOW_PROTOCOL to limit dangerous protocols
See git commit 33cfccbbf35a -- some protocols allow arbitrary command
execution as part of the URL. Instead of blindly allowing those,
whitelist the allowed URL protocols unless the user has already done so.

Bug: Issue 210
Change-Id: I6bd8e721aa5e3dab53ef28cfdc8fde33eb74ef76
2015-11-26 11:03:19 +09:00
Dan Willemsen
e1e0bd1f75 Check for broken links when updating linkfiles
If a linkfile is a broken link (destination does not exist), and it
needs to be updated, we didn't notice that it needed to be removed
first. Use lexists instead of exists to check for this condition.

Change-Id: I1f6a1f0193d3fd2b9f7a647836044997f6ab32eb
2015-11-18 16:51:51 -08:00
David Pursehouse
74cfd2709b Sync: Add option to prune refs during sync
By passing --prune to the sync command, the --prune option is
given to the `git fetch`, causing refs that no longer exist on
the remote to be removed.

Change-Id: I3cedacce14276d96ac2d5aabf2d07fd05e92bc02
2015-10-27 03:04:17 +00:00
Pascal Muetschard
c2a64ddffd A couple of fixes to the init command's -p option.
Adds windows as one of the allowed platforms flags.
Fixes -p foo to append 'platform-foo', instead of each letter (list.extend
expects a list and thus appends each char in the string, rather than the
string itself).

Change-Id: I73a92127ac29a32fc31b335cc54a246302904140
2015-10-22 13:28:20 -07:00
Dan Willemsen
745b4ad660 Fix gitc-init behavior
With gitc-init, a gitc client may be specified using '-c'. If we're
not currently in that client, we need to change directories so that
we don't affect the local checkout, and to ensure that repo is
checked out in the new client.

This also makes '-c' optional if already in a gitc client, to match
the rest of the init options.

Change-Id: Ib514ad9fd101698060ae89bb035499800897e9bd
2015-10-07 15:43:22 -07:00
David Pursehouse
4c5f74e452 Sync: Add HTTP Cookie File header on temporary cookie file
The .gitcookies file generated by googlesource.com does not have
the header:

 # (Netscape) HTTP Cookie File

which causes python's MozillaCookieJar.load to fail with the
error:

 "does not look like a Netscape format cookies file"

Prepend the expected header onto the generated cookie file.

We don't bother to check if the header already exists on the
file; repeating it does not cause any problem.

Bug: Issue 207
Change-Id: I7d39720a1d36a6aae00f70691156514ebc04e579
2015-10-02 11:12:05 +09:00
David Pursehouse
b1ad2190a2 Sync: Don't fail when git cookies can't be loaded
If the git cookies file fails to load, use a default
cookie jar instead.

Bug: Issue 207
Change-Id: I7cb326c204f2784ab4dbd13801b3186667af5b78
2015-10-02 11:04:01 +09:00
Simran Basi
f231db11a2 GITC: Add repo gitc-delete command.
repo gitc-delete deletes a GITC client and all the locally
saved sources. Useful for removing unnecessary clients and
recovering disk space.

Change-Id: Idf23addcea52b8713d268c34a7b37da0c5e5cd26
2015-10-01 21:05:17 +00:00
Dan Willemsen
79360640f4 Add GitcClientCommand class for GITC-specific commands
These won't show up as common commands in the help text unless in a GITC
client, and will refuse to execute.

Change-Id: Iffe82adcc9d6ddde9cb4b204f83ff018042bdab0
2015-09-29 13:46:34 -07:00
Dan Willemsen
7b01b2fd01 Merge "launcher: Update repo after applying clone.bundle" 2015-09-14 23:40:05 +00:00