Commit Graph

101 Commits

Author SHA1 Message Date
Ulrik Laurén
d0ca0f6814 Parse included files when reading git config files
Git config files may have an include tag pointing to another file.
The included file is not parsed unless “git config --list” is
explicitly told to follow includes by adding the argument ”--includes”.

This change add the "--includes" when parsing the global gitconfig file.

Change-Id: I892c9a3a748754c1eb8c9e220578305ca5850dd5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/264759
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Ulrik Laurén <ulrik.lauren@gmail.com>
2020-04-29 18:28:41 +00:00
Mike Frysinger
f841ca48c1 git_config: add support for repo-specific settings
This allows people to write ~/.repoconfig/config akin to ~/.gitconfig
and .repo/config akin to .git/config.  This allows us to add settings
specific to repo without mixing up git, and to persist in general.

Change-Id: I1c6fbe31e63fb8ce26aa85335349c6ae5b1712c6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255832
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2020-02-20 00:53:39 +00:00
Mike Frysinger
77b4397a73 git_config: add GetInt helper
Change-Id: Ic034ae2fd962299d1b352e597b391b6582ecf44b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/256052
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Jonathan Nieder <jrn@google.com>
2020-02-19 23:09:05 +00:00
Mike Frysinger
af1e5dea35 resort a few module imports to follow PEP8
All the stdlib imports are supposed to come before any local imports.

Change-Id: I10c0335ba2ff715fd34c9eb91bfe6560e904df08
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255593
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
2020-02-18 06:28:12 +00:00
David Pursehouse
16a5c3ac51 git_config: Stop using backslash to wrap lines
Unwrap one unnecessarily wrapped line, and use parentheses on
a wrapped condition instead of wrapping with backslashes.

Change-Id: I12679a0547dd822b15a6551e0f6c308239ff7b2d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254607
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 07:29:25 +00:00
David Pursehouse
819827a42d Fix blank line issues reported by flake8
- E301 expected 1 blank line
- E302 expected 2 blank lines
- E303 too many blank lines
- E305 expected 2 blank lines after class or function definition
- E306 expected 1 blank line before a nested definition

Fixed automatically with autopep8:

  git ls-files | grep py$ | xargs autopep8 --in-place \
    --select E301,E302,E303,E305,E306

Manually fix issues in project.py caused by misuse of block comments.

Change-Id: Iee840fcaff48aae504ddac9c3e76d2acd484f6a9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254599
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 06:36:40 +00:00
David Pursehouse
abdf750061 Fix indentation issues reported by flake8
- E121 continuation line under-indented for hanging indent
- E122 continuation line missing indentation or outdented
- E125 continuation line with same indent as next logical line
- E126 continuation line over-indented for hanging indent
- E127 continuation line over-indented for visual indent
- E128 continuation line under-indented for visual indent
- E129 visually indented line with same indent as next logical line
- E131 continuation line unaligned for hanging indent

Fixed automatically with autopep8:

  git ls-files | grep py$ | xargs autopep8 --in-place \
    --select E121,E122,E125,E126,E127,E128,E129,E131

Change-Id: Ifd95fb8e6a1a4d6e9de187b5787d64a6326dd249
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254605
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 06:36:22 +00:00
David Pursehouse
0ab95ba6d0 git_config: Unwrap unnecessarily wrapped line
Change-Id: I56806e8b9b09cd0f7fb834d7edc412682f2af1db
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254604
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 06:32:47 +00:00
David Pursehouse
54a4e6007a Fix various whitespace issues reported by pyflakes
- E201 whitespace after '['
- E202 whitespace before '}'
- E221 multiple spaces before operator
- E222 multiple spaces after operator
- E225 missing whitespace around operator
- E226 missing whitespace around arithmetic operator
- E231 missing whitespace after ','
- E261 at least two spaces before inline comment
- E271 multiple spaces after keyword

Fixed automatically with autopep8:

  git ls-files | grep py$ | xargs autopep8 --in-place \
    --select E201,E202,E221,E222,E225,E226,E231,E261,E271

Change-Id: I367113eb8c847eb460532c7c2f8643f33040308c
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254601
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
2020-02-12 06:00:16 +00:00
David Pursehouse
e5913ae410 Fix flake8 E251 unexpected spaces around keyword / parameter equals
Fixed automatically with autopep8:

  git ls-files | grep py$ | xargs autopep8 --in-place --select E251

Change-Id: I58009e1c8c91c39745d559ac919be331d4cd9e77
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254598
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
2020-02-12 05:17:08 +00:00
Mike Frysinger
ded477dbb9 git_config: fix encoding handling in GetUrlCookieFile
Make sure we decode the bytes coming from the subprocess.Popen as
we're treating them as strings.

Change-Id: I44100ca5cd94f68a35d489936292eb641006edbe
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253973
Reviewed-by: Jonathan Nieder <jrn@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2020-02-08 06:26:36 +00:00
Mike Frysinger
3164d40e22 use open context managers in more places
Use open() as a context manager to simplify the close logic and make
the code easier to read & understand.  This is also more Pythonic.

Change-Id: I579d03cca86f99b2c6c6a1f557f6e5704e2515a7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244734
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-11-12 03:44:39 +00:00
Mike Frysinger
8a11f6f24c rename local trace module
There is a standard Python "trace" module, so having a local trace.py
prevents us being able to import that.  Rename the module to avoid.

Change-Id: I23e29ec95a2204bb168a641323d05e76968d9b57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234832
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-08-27 07:08:52 +00:00
Mike Frysinger
1f2462e0d2 git_config: include project name in missing ref exception
When syncing in parallel, this exception is hard to trace back to
a specific repo as the relevant log line could have been pushed
out by other repos syncing code.

Change-Id: I382efeec7651e85622aa51e351134aef0148267f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233075
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Nasser Grainawi <nasser@codeaurora.org>
Tested-by: Mike Frysinger <vapier@google.com>
2019-08-03 16:15:48 +00:00
Mike Frysinger
1b9adab75a handle binary stream from urllib.request.urlopen
Python 3 returns bytes by default with urlopen.  Adjust our code to
handle that scenario and decode as necessary.

Bug: https://crbug.com/gerrit/10418
Change-Id: Icf4cd80e7ef92d71a3eefbc6113f1ba11c32eebc
2019-07-04 18:19:00 -04:00
Mike Frysinger
f601376e13 set default file encoding to utf-8
There's no reason to support any other encoding in these files.
This only affects the files themselves and not streams they open.

Bug: https://crbug.com/gerrit/10418
Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
2019-06-13 14:30:52 +00:00
Renaud Paquay
bed8b62345 Add support for long paths
* Add more file i/o wrappers in platform_utils to allow using
  long paths (length > MAX_PATH) on Windows.

* Paths using the long path syntax ("\\?\" prefix) should never
  escape the platform_utils API surface area, so that this
  specific syntax is not visible to the rest of the repo code base.

* Forward many calls from os.xxx to platform_utils.xxx in various place
  to ensure long paths support, specifically when repo decides to delete
  obsolete directories.

* There are more places that need to be converted to support long paths,
  this commit is an initial effort to unblock a few common use cases.

* Also, fix remove function to handle directory symlinks

Change-Id: If82ccc408e516e96ff7260be25f8fd2fe3f9571a
2018-10-22 08:16:35 -07:00
Dylan Deng
e469a0c741 fix some sync error while using python3
Change-Id: I70925e48756c356d48359679d8ad1b9e33a68595
2018-07-24 22:20:08 +08:00
David Pursehouse
65b0ba5aa0 Remove unused pylint suppressions
pylint is not used since bb5b1a0. The pyflakes cleanup mentioned in that
commit has not been done, but given that this project is no longer being
actively developed I don't think it's worth spending time doing it.

Leaving the pylint suppressions causes confusion because it leads people
to think that we are still using pylint.

Change-Id: If7d9f280a0f408c780f15915ffdb80579ae21f69
2018-07-24 22:20:08 +08:00
Daichi Ueura
ce7e02601c Take care of a tilde on cookie file path
This handles cookie file path like "~/.gitcookies".

Change-Id: I87ba120a940fff38073d520f83b70654e6a239ba
2018-02-26 08:53:08 +09:00
Renaud Paquay
010fed7711 Replace all os.remove calls
os.remove raises an exception when deleting read-only files on
Windows. Replace all calls with calls to platform_utils.remove,
which deals with deals with that issue.

Change-Id: I4dc9e0c9a36b4238880520c69f5075eca40f3e66
2017-08-31 13:49:36 -07:00
David Pursehouse
9d743397bf Merge "Fixed upload to remotes with the url ssh://hostname" 2017-08-30 15:11:22 +00:00
David Pursehouse
2c57d619bc Merge "Add option '--no-cert-checks' for 'upload' sub command." 2017-08-30 15:11:10 +00:00
Christian Koestlin
2ec2a5d64c Fixed upload to remotes with the url ssh://hostname
Change-Id: I1d0dd4d3f90eac45205f6f4ca98a29b0babdbc3f
2017-08-29 20:16:08 +00:00
Zac Livingston
9ead97bb51 When starting a branch, do not use a tag or change value for branch.merge
When starting a branch, branch.merge is set to project revision unless
the revision is a SHA1. In that case, branch.merge is set to dest_branch
if defined or manifest default revision otherwise. This special handling
allows repo upload to work when the project revision is a SHA1.

Extend the special handling to also happen when the project revision
is a tag value or a change value so that repo upload will work in those
case as well.

Change-Id: Iff81ece40e770cd02535e80dcb023564d42dcf47
2017-08-25 09:10:29 +09:00
Łukasz Gardoń
bed59cec5e Add option '--no-cert-checks' for 'upload' sub command.
This option allow to bypass verification ssl certification while
establishing connection with Gerrit to upload review.

Change-Id: If2e15f5a273c18a700eb5093ca8a4d5a4cbf80cd
2017-08-23 14:06:14 +02:00
Timo Lotterbach
eec726c6d8 Add option REPO_IGNORE_SSH_INFO to ignore ssh_info
This is required for setups, where Gerrit access using ssh is only available
for some networks.
For network without ssh access, repo will get ssh_info from Gerrit and
use ssh for communications - which will fail. To support this setup
we need to have an option to ignore the ssh_info provided by Gerrit and
use http(s).

Using git insteadOf as alternative results in the inability to add
reviewers using "repo upload --re=...", since the syntax of adding
reviewers differs for ssh and https. repo is assuming an ssh
connection and uses "git push --receive-pack=...", which will fail
since git silently uses https for push operation. repo must be aware
that https is used so it uses "git push remote ...:refs/for/...%r=..."
for upload.

Change-Id: Idd83baef0fb26ffcc9ac65e204b68d323ce177a1
2017-05-26 15:11:11 +02:00
Dan Willemsen
16889ba43d Revert "Repo: fall back to http, if ssh connection fails for http repos"
This reverts commit 488bf092d5.

Issue 230

Change-Id: I3a5725301f576e1a2ac499cb6daa631895115640
2016-09-22 16:40:27 +00:00
Steve Rae
d648045366 implement optional 'pushurl' in the manifest file
Allow the 'remote' element in the manifest file to define an optional
'pushurl' attribute which is passed into the .git/config file.

Change-Id: If342d299d371374aedc4440645798888869c9714
Signed-off-by: Steve Rae <steve.rae@raedomain.com>
2016-09-20 15:31:20 +00:00
David Pursehouse
628456833a Merge "Repo: improve error detection for new ssh connections" 2016-09-20 08:06:12 +00:00
Timo Lotterbach
488bf092d5 Repo: fall back to http, if ssh connection fails for http repos
if a gerrit server has ssh and https access enabled, but user access
(for some users) is limited to https, 'repo upload' command will fail
for them.

Gerrit returns a ssh configuration (gerrit/ssh_info), that does not work
for users limited to https.

With this patch repo will test, if the returned ssh configuration from
gerrit/ssh_info is working. if not, it will fall back to https for upload.

Change-Id: If98f472e994f350bf71f35610cd649b163f1ab33
2016-08-30 07:35:03 +00:00
Timo Lotterbach
05dc46b0e3 Repo: improve error detection for new ssh connections
this check can only detect errors that happen within 1 sec after launching
ssh. But this is typically enough to catch configuration issues like
'connection refused' or 'authentication failed'.

Change-Id: I00b6f62d4c2889b1faa6c820e49a198554c92795
2016-08-30 07:34:33 +00:00
Dan Willemsen
0745bb2657 Support smart-sync through persistent-http[s]
Use the same cookies and proxy that git traffic goes through for
persistent-http[s] to support authentication for smart-sync.

Change-Id: I20f4a281c259053a5a4fdbc48b1bca48e781c692
2015-08-19 10:22:11 -07:00
Anthony King
b1d1fd778d git_config: fix _SaveJson typo
Change-Id: I35ca2b3733e6d1508669f9a6690c6645c582912e
2015-06-04 00:22:23 +00:00
Yann Droneaud
936183a492 git_config: add support for remote '.'
As a fix for issue #149, this patch add support for the remote '.'
(local).

As an alias for the local repository, remote '.' is lacking a fetch =
config in .git/config.

Without such refspec, repo info --overview is not able to process a
local tracking branch.

v2: Check for name == '.' before checking if merge starts with refs/,
    since the case where it's not is invalid.

Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
Signed-off-by: Filipe Brandenburger <filbranden@google.com>

Change-Id: I8c8fd8602cd68baecb530301ae41d37d751ec85d
2015-03-06 13:23:27 -08:00
Conley Owens
7e12e0a2fa Support persistent-http(s) review urls
Change-Id: I8e0065685c968dfa9dc26bcdb6ee2fa14019c509
2014-10-23 15:42:09 -07:00
Nasser Grainawi
909d58b2e2 Support specifying non-HEADS refs as upstream
While not typical, some users might have an upstream that isn't in
the usual refs/heads/* namespace. There's no reason not to use
those refs as the value for the upstream attribute, so support
doing so.

Change-Id: I5b119f1135c3268c20e7c4084682e860d3ee1fb1
2014-10-09 13:41:51 -06:00
Chirayu Desai
303a82f33a Don't open non-binary files as binary
* Don't pen the git config file, and the git ".lock" file as binary.

Change-Id: I7b3939658456f2fd0a0500443cdd8d1ee1a4459d
2014-08-19 23:05:44 +05:30
Conley Owens
a6d258b84d Merge "Fix UrlInsteadOf to handle multiple strings" 2014-06-30 22:21:58 +00:00
Anthony King
85b24acd6a Use JSON instead of pickle
Use JSON as it is shown to be much faster than pickle.
Also clean up the loading and saving functions.

Change-Id: I45b3dee7b4d59a1c0e0d38d4a83b543ac5839390
2014-05-07 10:46:24 +01:00
Dan Willemsen
4e4d40f7c0 Fix UrlInsteadOf to handle multiple strings
For complex .gitconfig url rewrites, multiple insteadOf lines may be
used for a url. Search all of them for the right rewrite.

Change-Id: If5e9ecd054e86226924b0baf513801cd57c389cd
2014-03-06 21:04:18 -08:00
Conley Owens
2cd38a0bf8 Stop appending 'p/' to review urls
Gerrit no longer requires 'p/', and this causes unexpected behavior.
In this change we stop appending 'p/' to the urls.

Change-Id: I72c13bf838f4112086141959fb1af249f9213ce6
2014-02-04 15:32:29 -08:00
Steve Pucci
143d8a7249 Changes to support sso: repositories for upload
Change-Id: Iddf90d52f700a1f6462abe76d4f4a367ebb6d603
2014-01-31 07:39:44 -08:00
Chirayu Desai
0eb35cbe50 Fix some python3 encoding issues
* Add .decode('utf-8') where needed
* Add 'b' to `open` where needed, and remove where unnecessary

Change-Id: I0f03ecf9ed1a78e3b2f15f9469deb9aaab698657
2013-11-21 06:03:22 +00:00
Conley Owens
745a39ba3d Assume http upload if ssh_info cannot be parsed
When uploading a change for review, we sometimes request /ssh_info to
get the correct port number for uploading via ssh (regardless of
whether or not we intend to upload over ssh).

If we have trouble accessing /ssh_info (for authentication reasons,
etc), we now press on under the assumption that we will upload via http
instead of aborting.

Change-Id: Ica6bbeac1279e89424a903529649b7f4af0b6937
2013-09-27 19:15:34 +09:00
David Pursehouse
ecf8f2b7c8 Handle HTTPException when attempting to get ssh_info
The call to `urlopen` can raise `HTTPException`, but this is not
caught which results in a python Traceback.

Add handling of the exception.  Because `HTTPException` and its
derived classes do not have any message, print the name of the
exception in the error message instead.

Change-Id: Ic90fb4cc0e92702375cd976d4a03876c8ce8bffc
2013-05-25 08:07:52 +05:30
David Pursehouse
59bbb580e3 Move Python version checking to a separate module
Add a new module with methods for checking the Python version.

Instead of handling Python3 imports with try...except blocks, first
check the python version and then import the relevant modules.  This
makes the code a bit cleaner and will result in less diff when/if we
remove support for Python < 3 later.

Use the same mechanism to handle `input` vs. `raw_input` and add
suppression of pylint warnings caused by redefinition of the built-in
method `input`.

Change-Id: Ia403e525b88d77640a741ac50382146e7d635924
Also-by: Chirayu Desai <cdesai@cyanogenmod.org>
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
2013-05-23 07:28:53 +00:00
Chirayu Desai
217ea7d274 Some fixes for supporting python3
* Fix imports.
* Use python3 syntax.
* Wrap map() calls with list().
* Use list() only wherever needed.
  (Thanks Conley!)
* Fix dictionary iteration methods
  (s/iteritems/items/).
* Make use of sorted() in appropriate places
* Use iterators directly in the loop.
* Don't use .keys() wherever it isn't needed.
* Use sys.maxsize instead of sys.maxint

TODO:
* Make repo work fully with python3. :)

Some of this was done by the '2to3' tool [1], by
applying the needed fixes in a way that doesn't
break compatibility with python2.

Links:
[1]: http://docs.python.org/2/library/2to3.html

Change-Id: Ibdf3bf9a530d716db905733cb9bfef83a48820f7
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
2013-04-18 21:35:49 +05:30
David Pursehouse
8f62fb7bd3 Tidy up code formatting a bit more
Enable the following Pylint warnings:

  C0322: Operator not preceded by a space
  C0323: Operator not followed by a space
  C0324: Comma not followed by a space

And make the necessary fixes.

Change-Id: I74d74283ad5138cbaf28d492b18614eb355ff9fe
2012-11-14 12:09:38 +09:00
David Pursehouse
c1b86a2323 Fix inconsistent indentation
The repo coding style is to indent at 2 characters, but there are
many places where this is not followed.

Enable pylint warning "W0311: Bad indentation" and make sure all
indentation is at multiples of 2 characters.

Change-Id: I68f0f64470789ce2429ab11104d15d380a63e6a8
2012-11-14 11:38:57 +09:00