Commit Graph

1263 Commits

Author SHA1 Message Date
Conley Owens
4ccad7554b Fix substitution err for schemeless manifest urls
Previously, we used a regex that would only remove a phony string from
a url if it existed, but we recently replaced that with a slice.  This
change goes back to the previous behavior.

Change-Id: I8baf527be01c4b49d45b903b31a1cd6315563d5b
2015-04-29 10:45:37 -07:00
David Pursehouse
403b64edf4 Don't append branch to fetch spec when syncing to a mirror
Appending the branch to the fetch spec causes sync of a mirror to
fail for projects that don't have an explicit revision specified,
and don't have a branch of the same name as the default revision.

For example, a manifest defining a default revision:

 <default revision="master">

having a project without an explicit revision:

 <project name="path/to/project">

and not having a branch named "master", will cause repo sync to
fail for that project with the error:

 Couldn't find remote ref refs/heads/master

Modify the logic to not append the branch onto the fetch spec when
syncing to a mirror.

Change-Id: I5c4457bd125519abf27abe682dea62ad708978c9
2015-04-27 10:56:27 +09:00
Conley Owens
a38769cda8 Merge "forall: use a generator to map the Pool" 2015-04-08 17:59:58 +00:00
Conley Owens
44859d0267 Merge "status: lose dependence on StringIO" 2015-04-08 17:58:35 +00:00
Anthony King
6ad6dbefe7 forall: use a generator to map the Pool
Before, a list was generated, which is why there was a massive delay.

Using a generator will allow processes to start straight away

Change-Id: Ia325b0b340cc328c08c9bcc92a6709bbdaf6a664
2015-04-08 13:22:34 +01:00
David Pursehouse
33fe4e99f9 Remove deprecated include-ids setting from pylint config
Change-Id: Ie5ab21e434d24ff862bb5e0c263761370d71f56f
2015-04-07 11:10:17 +09:00
David Pursehouse
4214585073 Merge "Pylint and PEP8 fixes for color.py" 2015-04-07 02:06:49 +00:00
Anthony King
b51f07cd06 status: lose dependence on StringIO
buflist was being used, which isn't available in Python 3.

`Execute` was using StringIO to capture the output of `PrintWorkTreeStatus`,
only to redirect it straight to stdout.
Instead, just let `PrintWorkTreeStatus` do it's own thing directly to stdout.

for handling `_FindOrphans`, we swap StringIO for a list. Nothing was done
that needed a a file like object.

Change-Id: Ibdaae137904de66a5ffb590d84203ef0fe782d8b
2015-04-04 21:21:49 +01:00
Conley Owens
04f2f0e186 Maintain fully qualified tracking branches
When running repo branch, the git merge line (in many circumstances)
is set to the revision of the project specified in the manifest.  If
this is a branch name that is not fully-qualified, we will end up with
something like "merge = master" instead of "merge = refs/heads/master".
This change examines the revision if we are going to use that and
changes branch short names to fully qualified branch names.

Change-Id: Ie1be94fb8d45df8eeac44a47f729a3819a05fa81
2015-04-01 17:43:36 +00:00
Anthony King
cb07ba7e3d Resolve fetch urls more efficiently
Instead of using regex, append the netloc and relative
scheme lists with the custom scheme.
The schemes will only be appended when needed, instead
of passing X amount of regex replaces.

see http://bugs.python.org/issue18828 for more details.

Change-Id: I10d26d5ddc32e7ed04c5a412bdd6e13ec59eb70f
2015-03-31 20:12:44 +00:00
Anthony King
23ff7df6a7 use the max depth instead of unshallow
This allows the use of older versions of git

Change-Id: I88ea685066603af19896a791829355ddbfa91ffe
2015-03-30 21:54:26 +00:00
Anthony King
cc1b1a703d Revert "Change the min git version from 1.7.2 to 1.8.2"
This reverts commit 52b99aa91d.

Change-Id: I01d93704c92f7af1ca2b36dbc9509ee1290e2d3c
2015-03-30 21:53:25 +00:00
Anthony King
bdf7ed2301 Pylint and PEP8 fixes for color.py
Change-Id: I1a676e25957a7b5dd800d2585a2ec7fe75295668
2015-03-28 21:12:27 +00:00
John L. Villalovos
9c76f67f13 Always capture output for GitCommand
Switch the GitCommand program to always capture the output for stdout
and stderr.  And by default print the output while running.

The options capture_stdout and capture_stderr have effectively become
options to supress the printing of stdout and stderr.

Update the 'git fetch' to use '--progress' so that the progress messages
will be displayed.  git checks if the output location isatty() and if it
is not a TTY it will by default not print the progress messages.

Change-Id: Ifdae138e008f80a59195f9f43c911a1a5210ec60
2015-03-26 11:43:55 -07:00
Conley Owens
52b99aa91d Change the min git version from 1.7.2 to 1.8.2
This is needed for the --unshallow option of git fetch.

Change-Id: Ifdc5cec6130315c643924328fea425f1b94cb04a
2015-03-18 21:43:39 +00:00
Jonathan Nieder
9371979628 Revert "Implementation of manifest defined githooks"
This reverts commit 38e4387f8e.

A "repo init" followed by "repo sync" is meant to be as safe as
"git clone".  In particular it should not run arbitrary code provided
by the manifest owner.

It would still be nice to have support for manifest-defined git hooks
--- they'd just need a prompt like the upload RepoHook has.  Hopefully
a later change can bring them back.

Change-Id: I5ecd90fb5c2ed64f103d856d1ffcba38a47b062d
Signed-off-by: Jonathan Nieder <jrn@google.com>
2015-03-17 11:29:58 -07:00
Conley Owens
2086004261 Merge "Don't exit with error on HTTP 401 when downloading clone bundle" 2015-03-11 17:25:45 +00:00
Pascal Bach
2338788050 Don't exit with error on HTTP 401 when downloading clone bundle
If the server returns HTTP 401 (unauthorized) when attempting to
download clone bundle files, ignore it and continue, rather than
exiting with a fatal error.

Change-Id: I2c7ee03e149c354c7e4ad6ea1ebf266534778fe1
2015-03-11 07:43:40 +00:00
Jakub Vrana
0402cd882a Add space between project path and branch in repo status.
Currently, paths longer than 39 chars have no space after them so it looks
like this:

project path/branch master

Change-Id: I4c1bb13648ac099ade8a8d4ebafa04131571f842
2015-03-11 07:42:17 +00:00
Yann Droneaud
936183a492 git_config: add support for remote '.'
As a fix for issue #149, this patch add support for the remote '.'
(local).

As an alias for the local repository, remote '.' is lacking a fetch =
config in .git/config.

Without such refspec, repo info --overview is not able to process a
local tracking branch.

v2: Check for name == '.' before checking if merge starts with refs/,
    since the case where it's not is invalid.

Signed-off-by: Yann Droneaud <ydroneaud@opteya.com>
Signed-off-by: Filipe Brandenburger <filbranden@google.com>

Change-Id: I8c8fd8602cd68baecb530301ae41d37d751ec85d
2015-03-06 13:23:27 -08:00
Conley Owens
85e8267031 Merge "Implementation of manifest defined githooks" 2015-03-05 20:52:30 +00:00
John L. Villalovos
e30f46b957 Print stderr output from git command for RemoteFetch
The stderr output generated by git during a RemoteFetch was not being
printed.  This information is useful so print it.

Change-Id: I6e6ce12c4a57e5ca2359f76ce14f2fcbbc37a5ef
2015-02-25 14:29:28 -08:00
Conley Owens
e4978cfbe3 Ensure the repo project is never fetched with partial depth
If the repo project is synced with partial depth, then the tags
won't be fetched and users will be told the newest sha1 in the
stable branch isn't signed.

Change-Id: I107df97b4836b928c76aa33a700fa35d1705ae09
2015-02-10 14:44:05 -08:00
John L. Villalovos
126e298214 Handle case where 'git remote prune' needs to be run
Handle the case when this error occurs:
    error: some local refs could not be updated; try running
     'git remote prune origin' to remove any old, conflicting branches

This is usually caused by a reference getting changed from a file to a
directory.

For example:
  Initially someone creates a branch 'foo' and it is stored as:
    .git/refs/remotes/origin/foo

  Then later on it is decided to change the layout structure where 'foo'
  is a directory with branches below it:
    .git/refs/remotes/origin/foo/master

  The problem occurs when someone still has
  '.git/refs/remotes/origin/foo' on their system and does a repo sync.
  When this occurs the error message for needing to do a
  'git remote prune origin' occurs.

Now when doing a 'git fetch' if the error message from git says that a
'git remote prune' is needed, it will do the prune and then retry the
fetch.

Change-Id: I4c6f5aa6bd932f0ef7a39134400bedd52e82f633
Signed-off-by: John L. Villalovos <john.l.villalovos@intel.com>
2015-02-03 13:49:51 -08:00
Jimmie Wester
38e4387f8e Implementation of manifest defined githooks
When working within a team or corporation it is often
useful/required to use predefined git templates. This
change teaches repo to use a per-remote git hook template
structure.

The implementation is done as a continuation of the
existing projecthook functionality. The terminology is
therefore defined as projecthooks.

The downloaded projecthooks are stored in the .repo
directory as a metaproject separating them from the users
project forest.

The projecthooks are downloaded and set up when doing a
repo init and updated for each new repo init.

When downloading a mirror the projecthooks gits are
not added to the bare forest since the intention is to
ensure that the latest are used (allows for company policy
enforcement).

The projecthooks are defined in the manifest file in the
remote element as a subnode, the name refers to the
project name on the server referred to in the remote.
<remote name="myremote ..>
   <projecthook name="myprojecthookgit" revision="myrevision"/>
</remote>

The hooks found in the projecthook revision supersede
the stock hooks found in repo. This removes the need for
updating the projecthook gits for repo stock hook changes.

Change-Id: I6796b7b0342c1f83c35f4b3e46782581b069a561
Signed-off-by: Patrik Ryd <patrik.ryd@stericsson.com>
Signed-off-by: Ian Kumlien <ian.kumlien@gmail.com>
2015-02-03 16:01:15 +09:00
David Pursehouse
24245e0094 Merge "Add missing documentation of --current-branch option on sync command" 2015-01-31 12:44:45 +00:00
Conley Owens
db6f1b0884 Merge "Use depth flag when fetching" 2015-01-30 19:36:06 +00:00
David Pursehouse
f2fad61bde Add missing documentation of --current-branch option on sync command
Change-Id: I72d6e3d51241148c1df97bbad26338debb1fcb4e
2015-01-29 14:36:28 +09:00
Conley Owens
ee69084421 Merge "Handle shallow checkout of SHA1 pinned repos" 2015-01-28 20:29:37 +00:00
Conley Owens
d37d43f036 Merge "Don't delete hooks in .git/hooks" 2015-01-28 20:29:05 +00:00
Anthony King
7bdac71087 pylint fixes for project.py
Fix all the formatting warnings and unused variables

Change-Id: I17d88a23572303879530077f3a80451de5417fbb
2015-01-22 04:20:21 +00:00
Conley Owens
f97e8383a3 Use depth flag when fetching
Currently, we only use the depth flag when cloning.  The result is that when
new project history has merges, the entire history of the merged branch is
brought in and the project becomes unshallow very quickly.  --depth and
clone-depth are often used to save on space, not just network load, so this
seems less than ideal.

This change uses --depth on every fetch (when the user has depth specified),
not just the initial clone.  The result is that the given project stays
consistently shallow as opposed to growing over time, especially when merges
are involved.

Change-Id: Iac706cfdad4a555c72f9d9f1119195d38d91df12
2015-01-22 01:20:22 +00:00
Bertrand SIMONNET
3000cdad22 Handle shallow checkout of SHA1 pinned repos
When doing a shallow checkout SHA1 pinned repos with repo init --depth=1 and
repo sync -c, repo would try to fetch only some reference and fail if the exact
SHA1 repo was missing.
Instead, when depth is set, fetch only the specific commit.

Change-Id: If3f799d0e78c03faea47f796380bb5e367b11998
2015-01-21 14:14:23 -08:00
Mitchel Humpherys
b9d9efd394 Don't delete hooks in .git/hooks
We currently delete all hooks in .git/hooks for each project before
symlink'ing in the standard project hooks.  This can be annoying for
users who have installed custom git hooks.

There's no reason to delete all existing hooks.  Just rip out the
deletion code.

Change-Id: I5062a6cd20af700f6d6a17b11ad6c94853987c57
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
2015-01-15 22:49:08 -08:00
Dave Borowitz
497bde4de5 Respect --quiet when looking up bundle cookie file
Change-Id: I02a244132c49e4bb50ecda978974d6d2b220f6d1
2015-01-02 13:58:05 -08:00
Dave Borowitz
4abf8e6ef8 Save cookies back to jar when fetching clone.bundle
Change-Id: I3ef71b5e7f8ee1cda66057e46ae234866c7258c4
2015-01-02 13:57:14 -08:00
Dave Borowitz
137d0131bf Hold persistent proxy connection open while fetching clone.bundle
The persistent proxy may choose to present a per-process cookie file
that gets cleaned up after the process exits, to help with the fact
that libcurl cannot save cookies atomically when a cookie file is
shared across processes. We were letting this cleanup happen
immediately by closing stdin as soon as we read the configuration
option, resulting in a nonexistent cookie file by the time we use the
config option.

Work around this by converting the cookie logic to a context manager
method, which closes the process only when we're done with the cookie
file.

Change-Id: I12a88b25cc19621ef8161337144c1b264264211a
2015-01-02 13:57:13 -08:00
Conley Owens
42e679b9f6 Merge "add a global --color option" 2015-01-02 20:56:25 +00:00
Mike Frysinger
902665bce6 add a global --color option
If you want to turn off colors for commands, you have to manually adjust
the git config settings (in various locations).  If you're writing scripts
though, you often don't want to modify those locations.  Add a commandline
option to explicitly control things.

The default behavior is unchanged -- we still scan the config files.

Change-Id: I54a3fd8e1918bac180aadd7c7d3004f069b02522
2014-12-30 18:50:05 -05:00
Kris Giesing
c8d882ae2a Silence warnings about invalid clone.bundle files when quieted
The invalid clone.bundle file warning is not typically user actionable,
and can be confusing. So don't show it when -q flag is in effect.

Change-Id: If9fef4085391acf54b63c75029ec0e161c38eb86
2014-12-24 10:23:24 +09:00
Conley Owens
3eb87cec5c Revert "Check for existence of refs upon initial fetch"
This reverts commit 565480588d.

We are reverting this change for 2 reasons:

1) It introduced a bug for users using sync -c with a reference mirror.
2) The fetch specs have recently changed to cause git to properly fail
when we request a non-existent branch of a manifest, removing the need
for this change.

Change-Id: I0f63da9bfb40cf5ffafb7979f1b8c929a738fc7b
2014-11-10 23:49:32 +00:00
Max Liu
5fb8ed217c If revision is sha hash and dest-branch is defined, use it for starting branch
Change-Id: I538c7d216f72b87629b61aee547d374a398c95da
2014-10-27 12:25:05 +00:00
Conley Owens
7e12e0a2fa Support persistent-http(s) review urls
Change-Id: I8e0065685c968dfa9dc26bcdb6ee2fa14019c509
2014-10-23 15:42:09 -07:00
Conley Owens
7893b85509 Merge changes I1f71be22,I5b119f11
* changes:
  Always fetch the specific revision given
  Support specifying non-HEADS refs as upstream
2014-10-22 00:23:18 +00:00
Conley Owens
b4e50e67e8 Merge "upload: report names of uncommitted files" 2014-10-21 18:03:55 +00:00
Scott Anderson
0936aeab2c Exit 1 if repo download -c fails
Change-Id: I6985548bf87032b121eeccf858c4eeca1a60598c
2014-10-17 15:45:57 -04:00
Vadim Bendebury
14e134da02 upload: report names of uncommitted files
When there are uncommitted files in the tree, 'repo upload' stops to
ask if it is OK to continue, but does not report the actual names of
uncommitted files.

This patch adds plumbing to have the outstanding file names reported
if desired.

BUG=None
TEST=verified that 'repo upload' properly operates with the following
    conditions present in the tree:
    . file(s) modified locally
    . file(s) added to index, but not committed
    . files not known to git
    . no modified files (the upload proceeds as expected)

Change-Id: If65d5f8e8bcb3300c16d85dc5d7017758545f80d
Signed-off-by: Vadim Bendebury <vbendeb@chromium.org>
Signed-off-by: Vadim Bendebury <vbendeb@google.com>
2014-10-14 11:20:05 -07:00
Nasser Grainawi
04e52d6166 Always fetch the specific revision given
Don't assume the revision is in refs/heads/.

Change-Id: I1f71be222ed3ed940d2265aad43d1f2d601fc03a
2014-10-09 13:41:56 -06:00
Nasser Grainawi
909d58b2e2 Support specifying non-HEADS refs as upstream
While not typical, some users might have an upstream that isn't in
the usual refs/heads/* namespace. There's no reason not to use
those refs as the value for the upstream attribute, so support
doing so.

Change-Id: I5b119f1135c3268c20e7c4084682e860d3ee1fb1
2014-10-09 13:41:51 -06:00
Jeff Davidson
5cf16607d3 Allow selection of a target when using smart sync.
Change-Id: I02a24471b9b62dbba3773f22a289825bc566acd9
2014-10-02 10:17:44 -07:00