Commit Graph

1838 Commits

Author SHA1 Message Date
Anthony King
d58bfe5a58 Return a list rather than dict_values in XmlManifest.projects()
dict.values() produce dict_values objects rather than list objects.
Convert this to a list to maintain functionality with certain functions.

Change-Id: Ie76269e19f8d68479a1d7ae03aa965252d759a9e
2014-05-06 09:16:52 +01:00
Anthony King
70f6890352 Use exec() rather than execfile()
execfile() is not in Python 3.

Change-Id: I5af222340f13c1e8edaa820e7675d3e4d62a1689
2014-05-05 23:41:07 +01:00
Conley Owens
666d534636 Ensure HEAD is correct when skipping remote fetch
A recent optimization (2fb6466f79) skips
performing a remote fetch if we already know we have the sha1 we want.
However, that optimization skipped initialization steps that ensure HEAD
points to the correct sha1.  This change makes sure not to skip those
steps.

Here is an example of how to test this change:

"""""""""
url=<manifest url>
branch1=<branch name>
branch2=<branch name>
project=<project with revision set to different sha1 in each branch>

repo init -u $url -b $branch1 --mirror
repo sync $project
first=$(cd $project.git; git rev-parse HEAD)

repo init -b $branch2
repo sync $project
second=$(cd platform/build.git; git rev-parse HEAD)

if [[ $first == $second ]]
then
    echo 'problem!'
else
    echo 'no problem!'
fi
"""""""""
2014-05-01 13:20:32 -07:00
Conley Owens
f2af756425 Add 'shallow' gitfile to symlinks
This fixes the bug that kept clients from doing things like `git log`
in projects using the clone-depth feature.

Change-Id: Ib4024a7b82ceaa7eb7b8935b007b3e8225e0aea8
2014-04-30 11:34:00 -07:00
Conley Owens
544e7b0a97 Merge "Ignore clone-depth attribute when fetching to a mirror" 2014-04-24 21:21:02 +00:00
Jeff Hamilton
e0df232da7 Add linkfile support.
It's just like copyfile and runs at the same time as copyfile but
instead of copying it creates a symlink instead.  This is needed
because copyfile copies the target of the link as opposed to the
symlink itself.

Change-Id: I7bff2aa23f0d80d9d51061045bd9c86a9b741ac5
2014-04-22 14:35:47 -05:00
David Pursehouse
5a7c3afa73 Merge "Don't try to remove .repo if it doesn't exist" 2014-04-18 00:06:08 +00:00
David Pursehouse
9bc422f130 Ignore clone-depth attribute when fetching to a mirror
If a manifest includes projects with a clone-depth=1 attribute, and a
workspace is initialised from that manifest using the --mirror option,
any workspaces initialised and synced from the mirror will fail with:

  fatal: attempt to fetch/clone from a shallow repository

on the projects that had the clone-depth.

Ignore the clone-depth attribute when fetching from the remote to a
mirror workspace. Thus the mirror will be synched with a complete
clone of all the repositories.

Change-Id: I638b77e4894f5eda137d31fa6358eec53cf4654a
2014-04-16 11:00:40 +09:00
Mitchel Humpherys
e81bc030bb Add total count and iteration count to forall environment
For long-running forall commands sometimes it's useful to know which
iteration is currently running. Add REPO_I and REPO_COUNT environment
variables to reflect the current iteration count as well as the total
number of iterations so that the user can build simple status
indicators.

Example:

    $ repo forall -c 'echo $REPO_I / $REPO_COUNT; git gc'
    1 / 579
    Counting objects: 41, done.
    Delta compression using up to 8 threads.
    Compressing objects: 100% (19/19), done.
    Writing objects: 100% (41/41), done.
    Total 41 (delta 21), reused 41 (delta 21)
    2 / 579
    Counting objects: 53410, done.
    Delta compression using up to 8 threads.
    Compressing objects: 100% (10423/10423), done.
    Writing objects: 100% (53410/53410), done.
    Total 53410 (delta 42513), reused 53410 (delta 42513)
    3 / 579
    ...

Change-Id: I9f28b0d8b7debe423eed3b4bc1198b23e40c0c50
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
2014-03-31 13:08:26 -07:00
Mitchel Humpherys
eb5acc9ae9 Don't try to remove .repo if it doesn't exist
Part of the cleanup path for _Init is removing the .repo
directory. However, _Init can fail before creating the .repo directory,
so trying to remove it raises another exception:

    fatal: invalid branch name 'refs/changes/53/55053/4'
    Traceback (most recent call last):
      File "/home/mitchelh/bin/repo", line 775, in <module>
        main(sys.argv[1:])
      File "/home/mitchelh/bin/repo", line 749, in main
        os.rmdir(repodir)
    OSError: [Errno 2] No such file or directory: '.repo'

Fix this by only removing .repo if it actually exists.

Change-Id: Ia251d29e9c73e013eb296501d11c36263457e235
2014-03-12 15:11:27 -07:00
Mitchel Humpherys
26c45a7958 Make --no-tags work with -c
Currently, the --no-tags option is ignored if the user asks to only
fetch the current branch. There is no reason for this restriction. Fix
it.

Change-Id: Ibaaeae85ebe9955ed49325940461d630d794b990
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
2014-03-12 16:34:53 +09:00
David Pursehouse
68425f4da8 Fix indentation in project.py
Change-Id: I81c630536eaa54d5a25b9cb339a96c28619815ea
2014-03-11 14:55:52 +09:00
Dan Sandler
53e902a19b More verbose errors for NoManifestExceptions.
The old "manifest required for this command -- please run
init" is replaced by a more helpful message that lists the
command repo was trying to execute (with arguments) as well
as the str() of the NoManifestException. For example:

> error: in `sync`: [Errno 2] No such file or directory:
> 	'path/to/.repo/manifests/.git/HEAD'
> error: manifest missing or unreadable -- please run init

Other failure points in basic command parsing and dispatch
are more clearly explained in the same fashion.

Change-Id: I6212e5c648bc5d57e27145d55a5391ca565e4149
2014-03-11 05:33:43 +00:00
Dan Willemsen
4e4d40f7c0 Fix UrlInsteadOf to handle multiple strings
For complex .gitconfig url rewrites, multiple insteadOf lines may be
used for a url. Search all of them for the right rewrite.

Change-Id: If5e9ecd054e86226924b0baf513801cd57c389cd
2014-03-06 21:04:18 -08:00
bijia
093fdb6587 Add reviewers automatically from project's git config
The `review.URL.autocopy` setting sends email notification to the
named reviewers, but does not add them as reviewer on the uploaded
change.

Add a new setting `review.URL.autoreviewer`.  The named reviewers
will be added as reviewer on the uploaded change.

Change-Id: I3fddfb49edf346f8724fe15b84be8c39d43e7e65
Signed-off-by: bijia <bijia@xiaomi.com>
2014-03-04 00:51:30 +00:00
Chris AtLee
2fb6466f79 Don't fetch from remotes if commit id exists locally
In existing workspaces where the manifest specifies a commit id in the
manifest, we can avoid doing a fetch from the remote if we have the
commit locally. This substantially improves sync times for fully
specified manifests.

Change-Id: Ide216f28a545e00e0b493ce90ed0019513c61613
2014-03-03 10:17:03 +00:00
Conley Owens
724aafb52d Merge "Clean up duplicate logic in subcmds/sync.py." 2014-02-28 21:16:32 +00:00
Kwanhong Lee
ccd218cd8f Fix to mirror manifest when --mirror is given
Commit 8d201 "repo: Support multiple branches for the same project."
(Change id is I5e2f4e1a7abb56f9d3f310fa6fd0c17019330ecd) caused missing
mirroring manifest repository when 'repo sync' after 'repo init --mirror'.

When the function _AddMetaProjectMirror() is called to add two of
meta projects - git-repo itself and manifest repository to mirror,
it didn't add them into self._paths which has list of projects to be
sync'ed by 'repo sync'.

In addition, because member var of Project 'relpath' is used as a key
of self._paths, it should be set with proper value other than None.
Since this is only for meta projects which are not described in manifest
xml, 'relpath' is name of the projects.

Change-Id: Icc3b9e6739a78114ec70bf54fe645f79df972686
Signed-off-by: Kwanhong Lee <kwanhong.lee@windriver.com>
2014-02-20 11:07:23 +09:00
Julien Campergue
dd6542268a Add the "diffmanifests" command
This command allows a deeper diff between two manifest projects.
In addition to changed projects, it displays the logs of the
commits between both revisions for each project.

Change-Id: I86d30602cfbc654f8c84db2be5d8a30cb90f1398
Signed-off-by: Julien Campergue <julien.campergue@parrot.com>
2014-02-17 11:20:11 +00:00
David Pursehouse
baca5f7e88 Merge "Add error message for download -c conflicts" 2014-02-17 07:57:00 +00:00
David James
89ece429fb Clean up duplicate logic in subcmds/sync.py.
The fetch logic is now shared between the jobs == 1 and
jobs > 1 cases. This refactoring also fixes a bug where
opts.force_broken was not honored when jobs > 1.

Change-Id: Ic886f3c3c00f3d8fc73a65366328fed3c44dc3be
2014-02-14 16:14:32 +00:00
Conley Owens
565480588d Check for existence of refs upon initial fetch
When we do an initial fetch and have not specified any branch etc,
the following fetch command will not error:
git fetch origin --tags +refs/heads/*:refs/remotes/origin/*

In this change we make sure something got fetched and if not we report
an error.

This fixes the bug that occurs when we init using a bad manifest url and
then are unable to init again (because a manifest project has been
inited with no manifest).

Change-Id: I6f8aaefc83a1837beb10b1ac90bea96dc8e61156
2014-02-12 09:11:00 -08:00
Rob Ward
1829101e28 Add error message for download -c conflicts
Currently if you run repo download -c on a change and the cherry-pick
runs into a merge conflict a Traceback is produced:

rob@rob-i5-lm ~/Programming/repo_test/repo1 $ repo download -c repo1 3/1
From ssh://rob-i5-lm:29418/repo1
 * branch            refs/changes/03/3/1 -> FETCH_HEAD
error: could not apply 0c8b474... 2
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'
Traceback (most recent call last):
  File "/home/rob/Programming/git-repo/main.py", line 408, in <module>
    _Main(sys.argv[1:])
  File "/home/rob/Programming/git-repo/main.py", line 384, in _Main
    result = repo._Run(argv) or 0
  File "/home/rob/Programming/git-repo/main.py", line 143, in _Run
    result = cmd.Execute(copts, cargs)
  File "/home/rob/Programming/git-repo/subcmds/download.py", line 90, in Execute
    project._CherryPick(dl.commit)
  File "/home/rob/Programming/git-repo/project.py", line 1943, in _CherryPick
    raise GitError('%s cherry-pick %s ' % (self.name, rev))
error.GitError: repo1 cherry-pick 0c8b4740f876f8f8372bbaed430f02b6ba8b1898

This amount of error message is confusing to users and has the side effect
of the git message telling you the actual issue being ignored.

This change introduces a message stating that the cherry-pick couldn't
be completed removing the Traceback.

To reproduce the issue create a change that causes a conflict with one currently
in review and use repo download -c to cherry-pick the conflicting change.

Change-Id: I8ddf4e0c8ad9bd04b1af5360313f67cc053f7d6a
2014-02-11 18:19:04 +00:00
Conley Owens
1966133f8e Merge "Stop appending 'p/' to review urls" 2014-02-10 22:42:31 +00:00
David Pursehouse
f1027e23b4 Merge "Implement Kerberos HTTP authentication handler" 2014-02-05 00:58:53 +00:00
Conley Owens
2cd38a0bf8 Stop appending 'p/' to review urls
Gerrit no longer requires 'p/', and this causes unexpected behavior.
In this change we stop appending 'p/' to the urls.

Change-Id: I72c13bf838f4112086141959fb1af249f9213ce6
2014-02-04 15:32:29 -08:00
Conley Owens
1b46cc9b6d Merge "Changes to support sso: repositories for upload" 2014-02-04 21:19:07 +00:00
Carlos Aguado
1242e60bdd Implement Kerberos HTTP authentication handler
This commit implements a Kerberos HTTP authentication handler. It
uses credentials from a local cache to perform an HTTP authentication
negotiation using the GSSAPI.

The purpose of this handler is to allow the use Kerberos authentication
to access review endpoints without the need to transmit the user
password.

Change-Id: Id2c3fc91a58b15a3e83e4bd9ca87203fa3d647c8
2014-02-04 09:22:42 +01:00
Conley Owens
2d0f508648 Fix persistent-https relative url resolving
Previously, we would remove 'persistent-' then tack it on at the end
if it had been previously found.  However, this would ignore urljoin's
decision on whether or not the second path was relative.  Instead, we
were always assuming it was relative and that we didn't want to use
a different absolute url with a different protocol.

This change handles persistent-https:// in the same way we handled the
absense of an explicit protocol.  The only difference is that this time
instead of temporarily replacing it with 'gopher://', we use 'wais://'.

Change-Id: I6e8ad1eb4b911931a991481717f1ade01315db2a
2014-01-31 16:06:31 -08:00
Steve Pucci
143d8a7249 Changes to support sso: repositories for upload
Change-Id: Iddf90d52f700a1f6462abe76d4f4a367ebb6d603
2014-01-31 07:39:44 -08:00
Conley Owens
5db69f3f66 Update the version number on the repo launcher
The repo launcher version needs to be updated so some users can take
advantage of the more robust version number parsing.

Change-Id: Ibcd8036363311528db82db2b252357ffd21eb59b
2014-01-30 16:00:35 -08:00
Conley Owens
ff0a3c8f80 Share git version parsing code with wrapper module
'repo' and 'git_command.py' had their own git version parsing code.
This change shares that code between the modules.  DRY is good.

Change-Id: Ic896d2dc08353644bd4ced57e15a91284d97d54a
2014-01-30 15:18:56 -08:00
Conley Owens
094cdbe090 Add wrapper module
This takes the wrapper importing code from main.py and moves it into
its own module so that other modules may import it without causing
circular imports with main.py.

Change-Id: I9402950573933ed6f14ce0bfb600f74f32727705
2014-01-30 15:17:09 -08:00
Conley Owens
148a84de0c Respect version hyphenation
The last change regarding version parsing lost handling of version
hyphenation, this restores that.  In otherwords,
1.1.1-otherstuff is parsed as (1,1,1) instead of (1,1,0)

Change-Id: I3753944e92095606653835ed2bd090b9301c7194
2014-01-30 13:53:55 -08:00
Conley Owens
1c5da49e6c Handle release candidates in git version parsing
Right now repo chokes on git versions like "1.9.rc1".  This change
treats 'rc*' as a '0'.

Change-Id: I612b7b431675ba7415bf70640a673e48dbb00a90
2014-01-30 13:26:50 -08:00
David James
b8433dfd2f repo: Fix 'remove-project' regression with multiple projects.
In CL:50715, I updated repo to handle multiple projects, but the
remove-projects code path was not updated accordingly. Update it.

Change-Id: Icd681d45ce857467b584bca0d2fdcbf24ec6e8db
2014-01-30 10:14:54 -08:00
Conley Owens
f2fe2d9b86 Properly iterate through values
the value of Manifest.projects has changed from being the dictionary
to the values of the dictionary.  Here we handle this change
correctly on a PostRepoUpgrade.

From a `git diff v1.12.7 -- manifest_xml.py`:
+  @property
   def projects(self):
     self._Load()
-    return self._projects
+    return self._paths.values()

self._paths does contain the projects according to this line of
manifest_xml.py:
484      self._paths[project.relpath] = project

Change-Id: I141f8d5468ee10dfb08f99ba434004a307fed810
2014-01-29 13:57:22 -08:00
Conley Owens
c9877c7cf6 Merge "Only fetch current branch on shallow clients" 2014-01-29 21:12:34 +00:00
Shawn Pearce
69e04d8953 Only fetch current branch on shallow clients
Fetching a new branch on a shallow client may download the entire
project history, as the depth parameter is not passed to git
fetch. Force the fetch to only download the current branch.

Change-Id: Ie17ce8eb5e3487c24d90b2cae8227319dea482c8
2014-01-29 12:48:54 -08:00
David Pursehouse
f1f1137d61 Merge "Don't backtrace when current branch is not uploadable." 2014-01-14 00:41:35 +00:00
David Pursehouse
f77ef2edb0 Merge "hooks/pre-auto-gc: fix AC detection on OSX Maverick" 2014-01-10 02:50:53 +00:00
Conley Owens
e695338e21 Merge "repo: Support multiple branches for the same project." 2014-01-10 01:20:13 +00:00
Conley Owens
bd80f7eedd Merge "Canonicalize project hooks path before use" 2014-01-09 02:11:10 +00:00
David James
bf79c6618e Fix os.mkdir race condition.
This code checks whether a dir exists before creating it. In between the
check and the mkdir call, it is possible that another process will have
created the directory. We have seen this bug occur many times in
practice during our 'repo init' tests.

Change-Id: Ia47d39955739aa38fd303f4e90be7b4c50d9d4ba
2013-12-26 14:59:00 -08:00
Conley Owens
f045d49a71 Merge "Add --archive option to init to sync using git archive" 2013-12-18 17:44:59 +00:00
Pawit Pornkitprasan
719757d6a8 hooks/pre-auto-gc: fix AC detection on OSX Maverick
The output of pmset has been changed to "Now drawing from 'AC Power'"

Change-Id: Id425d3bcd6a28656736a6d2c3096623a3ec053cc
2013-12-17 09:48:20 +07:00
Warren Turkal
011d4f426c Don't backtrace when current branch is not uploadable.
The backtrace currently occurs when one uses the "--cbr" argument with
the repo upload subcommand if the current branch is not tracking an
upstream branch. There may be other cases that would backtrace as well,
but this is the only one I found so far.

Change-Id: Ie712fbb0ce3e7fe3b72769fca89cc4c0e3d2fce0
2013-12-11 23:24:01 -08:00
Warren Turkal
53d6a7b895 Fix error in xml manifest doc.
The docs on the annotations say that zero or more may exist as a child
of a project, so that means that a "*" instead of a "?" should be used.

Change-Id: Iff855d003dfb05cd980f285a237332914e1dad70
2013-12-10 15:30:03 -08:00
Julien Campergue
335f5ef4ad Add --archive option to init to sync using git archive
This significantly reduces sync time and used brandwidth as only
a tar of each project's revision is checked out, but git is not
accessible from projects anymore.

This is relevant when git is not needed in projects but sync
speed/brandwidth may be important like on CI servers when building
several versions from scratch regularly for example.

Archive is not supported over http/https.

Change-Id: I48c3c7de2cd5a1faec33e295fcdafbc7807d0e4d
Signed-off-by: Julien Campergue <julien.campergue@parrot.com>
2013-12-10 08:27:07 +00:00
Jesse Hall
672cc499b9 Canonicalize project hooks path before use
If the top-level .repo directory is moved somewhere else (e.g. a
different drive) and replaced with a symlink, _InitHooks() will create
broken symlinks. Resolving symlinks before computing the relative path
for the symlink keeps the path within the repo tree, so the tree can
be moved anywhere.

Change-Id: Ifa5c07869e3477186ddd2c255c6c607f547bc1fe
2013-12-03 09:02:16 -08:00