Commit Graph

167 Commits

Author SHA1 Message Date
Dan Willemsen
fee390eea2 launcher: Update repo after applying clone.bundle
If the clone.bundle is out of date, repo may be installed with an old
version. It will upgrade with the next sync a day later, or when "repo
selfupdate" is run.

This behavior was added to normal project downloads, but was never added
to the repo launcher.

Change-Id: Ib04bef3a658c98fe1b6c53b3e8d0067165a5e3f7
2015-09-02 12:45:19 -07:00
Dan Willemsen
9ff2ece6ab gitc: Improve help visibility
This improves the visiblity of gitc-init if we can get the gitc config,
and hides it otherwise.

Change-Id: I82830b0b07c311e8c74397ba79eb4c361f8b6fb5
2015-09-01 12:23:56 -07:00
Dan Willemsen
2487cb7b2c Fix gitc check if gitc isn't installed
This was doing cwd.startswith(''), which is always true.

Change-Id: Icc059c09492b31e2d7651e4a595bda783c5abc47
2015-08-31 15:59:54 -07:00
Simran Basi
8ce5041596 GITC: Pull GITC Manifest Dir from the config.
Updates the repo launcher and gitc_utils to pull the manifest
directory location out of the gitc config file.

Change-Id: Id08381b8a7d61962093d5cddcb3ff6afbb13004b
2015-08-31 21:39:17 +00:00
Simran Basi
1efc2b4a01 GITC: Add gitc-init subcommand to repo.
Adds the new gitc-init command to set up a GITC client. Gitc-init
sets up the client directory and calls repo init within it. Once
the repo is initialized, then generates a GITC manifest file
by using git ls-remote on each project and retrieving the HEAD SHA
to use as the revision attribute.

Gitc-init inherits from and has all the options as repo init.

Change-Id: Icd7e47e90eab752a77de7c80ebc98cfe16bf6de3
2015-08-12 16:22:14 -07:00
John Törnblom
d3ddcdbd8a Ignore clone.bundle on HTTP 501, i.e. Not Implemented
Change-Id: I03ee003d3bd5d0684a31bdf7961a55a511dfa0e2
2015-08-12 20:12:51 +02:00
Anthony King
cc1b1a703d Revert "Change the min git version from 1.7.2 to 1.8.2"
This reverts commit 52b99aa91d.

Change-Id: I01d93704c92f7af1ca2b36dbc9509ee1290e2d3c
2015-03-30 21:53:25 +00:00
Conley Owens
52b99aa91d Change the min git version from 1.7.2 to 1.8.2
This is needed for the --unshallow option of git fetch.

Change-Id: Ifdc5cec6130315c643924328fea425f1b94cb04a
2015-03-18 21:43:39 +00:00
Pascal Bach
2338788050 Don't exit with error on HTTP 401 when downloading clone bundle
If the server returns HTTP 401 (unauthorized) when attempting to
download clone bundle files, ignore it and continue, rather than
exiting with a fatal error.

Change-Id: I2c7ee03e149c354c7e4ad6ea1ebf266534778fe1
2015-03-11 07:43:40 +00:00
Mani Chandel
f75870beac Change implementation of cleanup in case of clone failure during "repo init"
Fix includes:
1. It deletes only .repo/repo instead of the whole .repo repository.

Bug: Issue 161
Change-Id: I1ab8caa7538fec5e6206d1b029f63bd3f60dedcd
2014-09-03 13:56:04 +05:30
Anthony King
6efdde9f6e Prevent warning twice about Python 3 usage
Only warn about using Python 3 when running the repo script directly.
This prevents the user being warned twice.

Change-Id: I2ee51ea2fa0127ea310598320e460ec9f38c6488
2014-05-06 12:44:22 +00:00
Mitchel Humpherys
eb5acc9ae9 Don't try to remove .repo if it doesn't exist
Part of the cleanup path for _Init is removing the .repo
directory. However, _Init can fail before creating the .repo directory,
so trying to remove it raises another exception:

    fatal: invalid branch name 'refs/changes/53/55053/4'
    Traceback (most recent call last):
      File "/home/mitchelh/bin/repo", line 775, in <module>
        main(sys.argv[1:])
      File "/home/mitchelh/bin/repo", line 749, in main
        os.rmdir(repodir)
    OSError: [Errno 2] No such file or directory: '.repo'

Fix this by only removing .repo if it actually exists.

Change-Id: Ia251d29e9c73e013eb296501d11c36263457e235
2014-03-12 15:11:27 -07:00
Conley Owens
5db69f3f66 Update the version number on the repo launcher
The repo launcher version needs to be updated so some users can take
advantage of the more robust version number parsing.

Change-Id: Ibcd8036363311528db82db2b252357ffd21eb59b
2014-01-30 16:00:35 -08:00
Conley Owens
ff0a3c8f80 Share git version parsing code with wrapper module
'repo' and 'git_command.py' had their own git version parsing code.
This change shares that code between the modules.  DRY is good.

Change-Id: Ic896d2dc08353644bd4ced57e15a91284d97d54a
2014-01-30 15:18:56 -08:00
David James
bf79c6618e Fix os.mkdir race condition.
This code checks whether a dir exists before creating it. In between the
check and the mkdir call, it is possible that another process will have
created the directory. We have seen this bug occur many times in
practice during our 'repo init' tests.

Change-Id: Ia47d39955739aa38fd303f4e90be7b4c50d9d4ba
2013-12-26 14:59:00 -08:00
Julien Campergue
335f5ef4ad Add --archive option to init to sync using git archive
This significantly reduces sync time and used brandwidth as only
a tar of each project's revision is checked out, but git is not
accessible from projects anymore.

This is relevant when git is not needed in projects but sync
speed/brandwidth may be important like on CI servers when building
several versions from scratch regularly for example.

Archive is not supported over http/https.

Change-Id: I48c3c7de2cd5a1faec33e295fcdafbc7807d0e4d
Signed-off-by: Julien Campergue <julien.campergue@parrot.com>
2013-12-10 08:27:07 +00:00
Chirayu Desai
5d6cb80b8f Allow using repo with python3
* Switching from python2 to python3 in the same workspace isn't
  currently supported, due to a change in the pickle version (which
  isn't supported by python2)
* Basic functionality does work with python3, however not everything
  is expected to

Change-Id: I4256b5a9861562d0260b503f972c1569190182aa
2013-11-21 18:44:52 +05:30
Conley Owens
53263d873d Merge "repo: use explicit Python executable to run main.py" 2013-10-10 18:42:59 +00:00
Conley Owens
5e0ee14575 Do not use print_function from __future__
Python 2.4 and 2.5 do not have a print_function available, so we need a
compatible print function for displaying an error message when the user
has an older version of Python.

Change-Id: I54d7297be98bb53970e873b36c6605e6dad386c3
2013-09-27 09:32:02 +09:00
anatoly techtonik
3a2a59eb87 repo: use explicit Python executable to run main.py
Small step to support non-POSIX platforms.

Change-Id: I3bdb9c82c2dfbacb1da328caaa1a406ab91ad675
2013-09-21 20:03:57 +03:00
Conley Owens
bc0308478b Update gpg key for cco3@android.com
cco3@android.com has a new gpg key, so this needs to be updated in the
repo scripts so that he can sign updates.

Change-Id: I9f058263b35bd027502d6e3b814d7aeb801a1e6e
2013-07-01 11:22:01 -07:00
David Pursehouse
59bbb580e3 Move Python version checking to a separate module
Add a new module with methods for checking the Python version.

Instead of handling Python3 imports with try...except blocks, first
check the python version and then import the relevant modules.  This
makes the code a bit cleaner and will result in less diff when/if we
remove support for Python < 3 later.

Use the same mechanism to handle `input` vs. `raw_input` and add
suppression of pylint warnings caused by redefinition of the built-in
method `input`.

Change-Id: Ia403e525b88d77640a741ac50382146e7d635924
Also-by: Chirayu Desai <cdesai@cyanogenmod.org>
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
2013-05-23 07:28:53 +00:00
Chirayu Desai
51813dfed1 repo: add rudimentary version checking
Change-Id: I957775c7ce0821971cc2320597e1a7a31950bcf3
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
2013-04-17 13:43:10 +05:30
David Holmer
0a1c6a1c16 Special handling for manifest group "default"
Change Details:
* Make "default" a special manifest group that matches any project that
  does not have the special project group "notdefault"
* Use "default" instead of "all,-notdefault" when user does not specify
  manifest group
* Expand -g option help to include example usage of manifest groups

Change Benefits:
* Allow a more intuitive and expressive manifest groups specification:
  * "default" instead of "all,-notdefault"
  * "default,foo" instead of "all,-notdefault,foo"
  * "default,-foo" instead of "all,-notdefault,-foo"
  * "foo,-default" which has no equivalent
* Default manifest groups behavior can be restored by the command
  'repo init -g default'. This is significantly more intuitive than the
  current equivalent command 'repo init -g all,-notdefault'.

Change-Id: I6d0673791d64a650110a917c248bcebb23b279d3
2013-04-03 22:27:45 +00:00
David Pursehouse
87b9d9b4f2 Don't exit with error on HTTP 403 when downloading clone bundle
If the server returns HTTP 403 (forbidden) when attempting to
download clone bundle files, ignore it and continue, rather than
exiting with a fatal error.

Change-Id: Icf78cba0332b51b0e7b622f7c7924369b551b6f6
2013-01-31 21:12:08 +09:00
David Pursehouse
3794a78b80 Sync help text in repo from init.py
Change Ia6032865f9296b29524c2c25b72bd8e175b30489 improved the
help text for the init command, but the same improvement was not made
in repo.

Change-Id: Idc34e479b5237137b90e8b040824776e4f7883b0
2012-11-15 06:21:24 +09:00
David Pursehouse
8f62fb7bd3 Tidy up code formatting a bit more
Enable the following Pylint warnings:

  C0322: Operator not preceded by a space
  C0323: Operator not followed by a space
  C0324: Comma not followed by a space

And make the necessary fixes.

Change-Id: I74d74283ad5138cbaf28d492b18614eb355ff9fe
2012-11-14 12:09:38 +09:00
David Pursehouse
98ffba1401 Fix: "Statement seems to have no effect"
Pylint raises an error on the call:

  print

Change it to:

 print()

Change-Id: I507e1b3dd928fa6c32ea7e86260fb3d7b1428e6f
2012-11-14 11:38:57 +09:00
Sarah Owens
cecd1d864f Change print statements to work in python3
This is part of a series of changes to introduce Python3 support.

Change-Id: I373be5de7141aa127d7debdbce1df39148dbec32
2012-11-13 17:33:56 -08:00
David Pursehouse
685f080d62 More code style cleanup
Clean up a few more unnecessary usages of lambda in `repo` that were missed
in the previous sweep that only considered files ending in .py.

Remove a duplicate import.

Change-Id: I03cf467a5630cbe4eee6649520c52e94a7db76be
2012-11-14 08:34:39 +09:00
David Pursehouse
8898e2f26d Remove magic hack
It should be assumed that on modern development environments, python
is accessible to /usr/bin/env

Change the shebang as necessary and remove the magic hack.

This also means losing the -E option on the call to python, so that
PYTHONPATH and PYTHONHOME will be respected and local configuration
problems in those vars would be noticed

Change-Id: I6f0708ca7693f05a4c3621c338f03619563ba630
2012-11-14 08:17:11 +09:00
Sarah Owens
a6053d54f1 Change usages of xrange() to range()
In Python3, range() creates a generator rather than a list.

None of the parameters in the ranges changed looked large enough
to create an impact in memory in Python2.  Note: the only use of
range() was for iteration and did not need to be changed.

This is part of a series of changes to introduce Python3 support.

Change-Id: I50b665f9296ea160a5076c71f36a65f76e47029f
2012-11-01 13:36:50 -07:00
Conley Owens
e072a92a9b Merge "Use python3 urllib when urllib2 not available" 2012-11-01 10:13:34 -07:00
Conley Owens
7601ee2608 Merge "Use 'stat' package instead of literals for mkdir()" 2012-11-01 10:01:18 -07:00
Sarah Owens
1f7627fd3c Use python3 urllib when urllib2 not available
This is part of a series of changes to introduce Python3 support.

Change-Id: I605b145791053c1f2d7bf3c907c5a68649b21d12
2012-10-31 14:26:48 -07:00
Sarah Owens
60798a32f6 Use 'stat' package instead of literals for mkdir()
This is part of a series of changes to introduce Python3 support.

Change-Id: Ic988ad181d32357d82dfa554e70d8525118334c0
2012-10-31 09:11:16 -07:00
Conley Owens
c5aa4d3528 Update minimum git version to 1.7.2
We now use the -c flag which was introduced in git 1.7.2.

Change-Id: I9195c0f6ac9fa63e783a03628049fe2c67d258ff
2012-10-26 11:34:11 -07:00
Conley Owens
55e4d464a7 Add a PGP key for cco3@android.com
This change adds a PGP key to allow cco3@android.com to sign releases.

Change-Id: I18a70c8b7d8f272dd1aad9d6b2e4a237ef35af33
2012-10-26 07:03:59 -07:00
Conley Owens
c9129d90de Update PGP keys during _PostRepoUpgrade in sync
Previously, if a key was added, a client wouldn't add the key during
the sync step.  This would cause issues if a new key were added and a
subsequent release were signed by that key.

Change-Id: I4fac317573cd9d0e8da62aa42e00faf08bfeb26c
2012-10-25 17:48:35 -07:00
Sarah Owens
a5be53f9c8 Use modern Python exception syntax
"except Exception as e" instead of "except Exception, e"

This is part of a transition to supporting Python 3.  Python >= 2.6
support "as" syntax.

Note: this removes Python 2.5 support.

Change-Id: I309599f3981bba2b46111c43102bee38ff132803
2012-10-23 21:35:59 -07:00
Mickaël Salaün
2f6ab7f5b8 Rename "dir" variables
The variable name "dir" conflicts with the name of a Python built-in
function: http://docs.python.org/library/functions.html#dir

Change-Id: I850f3ec8df7563dc85e21f2876fe5e6550ca2d8f
2012-10-10 08:30:15 +02:00
David Pursehouse
e00aa6b923 Clean up imports
manifest_xml: import `HEAD` and `R_HEADS` from correct module
version: import `HEAD` from correct module

`HEAD` and `R_HEADS` should be imported from the git_refs module,
where they are originally defined, rather than from the project
module.

repo: remove unused import of readline

cherry_pick: import standard modules on separate lines
smartsync: import subcmd modules explicitly from subcmd

Use:
  `import re
  import sys`
and
  `from subcmds.sync import Sync`

Instead of:
  `import sys, re`
and
  `from sync import Sync`

Change-Id: Ie10dd6832710939634c4f5c86b9ba5a9cd6fc92e
2012-09-18 09:54:57 +02:00
Shawn O. Pearce
3f61950f01 Use gerrit.googlesource.com/git-repo as the default URL
This is basically the same repository, but may be slightly more
up-to-date than the one on code.google.com/p/git-repo.

Change-Id: I5c99539f53231958eefb6993f00997c9adf0a3c9
2012-06-05 07:57:24 -07:00
Shawn O. Pearce
4fd38ecc3a Detect git is not installed
Fix detection for Git not being in $PATH during the initial
run of `repo init` in a new directory.

Change-Id: I2b1fcce1fb8afc47271f5c3bd2a28369009b2fb7
2012-06-05 07:56:09 -07:00
Conley Owens
d21720db31 Add a --platform flag
Projects may optionally specify their platform
(eg, groups="platform-linux" in the manifest).

By default, repo will automatically detect the platform. However,
users may specify --platform=[auto|all|linux|darwin].

Change-Id: Ie678851fb2fec5b0938aede01f16c53138a16537
2012-04-23 12:50:00 -07:00
Conley Owens
971de8ea7b Refine groups functionality
Every project is in group "default".  "-default" does not remove
it from this project.  All group names specified in the manifest
are positive names as opposed to a mix of negative and positive.

Specified groups are resolved in order.  If init is supplied with
--groups="group1,-group2", the following describes the project
selection when syncing:

  * all projects in "group1" will be added, and
  * all projects in "group2" will be removed.

Change-Id: I1df3dcdb64bbd4cd80d675f9b2d3becbf721f661
2012-04-23 12:39:05 -07:00
Colin Cross
5acde75e5d Add manifest groups
Allows specifying a list of groups with a -g argument to repo init.
The groups act on a group= attribute specified on projects in the
manifest.
All projects are implicitly labelled with "default" unless they are
explicitly labelled "-default".
Prefixing a group with "-" removes matching projects from the list
of projects to sync.
If any non-inverted manifest groups are specified, the default label
is ignored.

Change-Id: I3a0dd7a93a8a1756205de1d03eee8c00906af0e5
Reviewed-on: https://gerrit-review.googlesource.com/34570
Reviewed-by: Shawn Pearce <sop@google.com>
Tested-by: Shawn Pearce <sop@google.com>
2012-04-13 09:46:00 -07:00
Xiaodong Xu
ae0a36c9a5 Add support for Apache Digest authentication for repo init.
repo tool supports only Basic authentication for now. For those
who want to use this tool to manage their own projects, in case
the administrator has configured the Apache server with Digest
authentication method, users will fail to be authenticated when
they run the command 'repo init'.
Add the digest authentication password manager to the handler
list will fix this issue.

Since Git HTTP protocol will require the user be authenticated
for fetch operation first before pushing commits to the remote,
it is unlikely for the administrator to implement anonymous
read (aka pull) access and write access (aka push) for
authenticated user. Both read and write have to be authenticated.
Be aware that the user may have to add an extra line in his
~/.netrc file:
-------------------
account example.com
-------------------
where 'example.com' is the realm for Apache Digest authentication.

Change-Id: I76eb27b205554426d9ce1965deaaf727b87916cd
Signed-off-by: Xiaodong Xu <stid.smth@gmail.com>
2012-03-14 15:01:34 -07:00
Shawn O. Pearce
34fb20f67c Revert "Default repo manifest settings in git config"
This reverts commit ee1c2f5717.

This breaks a lot of buildbot systems. Rolling it back for now
until we can understand what the breakage was and how to fix it.
2011-11-30 13:41:02 -08:00
Victor Boivie
841be34968 Don't prompt the user for name/email unless necessary
If the user has already configured a workspace, use these values
when re-running 'repo init'.

Otherwise, if the user has global name and e-mail set, use these.

It's always possible to override this and be prompted by specifying
--config-name when running 'repo init'.

Change-Id: If45f0e4b14884071439fb02709dc5cb53f070f60
2011-11-29 14:31:56 -08:00
Victor Boivie
ee1c2f5717 Default repo manifest settings in git config
A default manifest URL can be specified using:
  git config --global repo-manifest.<id>.url <url>

A default manifest server can be specified using:
  git config --global repo-manifest.<id>.server <url>

A default git mirror reference can be specified using:
  git config --global repo-manifest.<id>.reference <path>

This will allow the user to use 'repo init -u <id>' as
a shorter alternative to specifying the full URL.

Also, manifest server will not have to be specified in the
manifest XML and the reference will not have to be specified
on the command line. If they are, they will override these
default values however.

Change-Id: Ifdbc160bd5909ec7df9efb0c5d7136f1d9351754
Signed-off-by: Victor Boivie <victor.boivie@sonyericsson.com>
2011-11-29 14:24:58 -08:00
Shawn O. Pearce
f322b9abb4 sync: Support downloading bundle to initialize repository
An HTTP (or HTTPS) based remote server may now offer a 'clone.bundle'
file in each repository's Git directory. Over an http:// or https://
remote repo will first ask for '$URL/clone.bundle', and if present
download this to bootstrap the local client, rather than relying
on the native Git transport to initialize the new repository.

Bundles may be hosted elsewhere. The client automatically follows a
HTTP 302 redirect to acquire the bundle file. This allows servers
to direct clients to cached copies residing on content delivery
networks, where the bundle may be closer to the end-user.

Bundle downloads are resumeable from where they last left off,
allowing clients to initialize large repositories even when the
connection gets interrupted.

If a bundle does not exist for a repository (a HTTP 404 response
code is returned for '$URL/clone.bundle'), the native Git transport
is used instead. If the client is performing a shallow sync, the
bundle transport is not used, as there is no way to embed shallow
data into the bundle.

Change-Id: I05dad17792fd6fd20635a0f71589566e557cc743
Signed-off-by: Shawn O. Pearce <sop@google.com>
2011-09-28 10:07:36 -07:00
Shawn O. Pearce
44da16e8a0 Change default REPO_URL to code.google.com
Change-Id: If7700daf96fb8f3ee449e5774017272ef31b4b44
2011-09-05 14:16:49 -07:00
Doug Anderson
49cd59bc86 Add --depth option to main repo wrapper.
See related repo change:
  https://review.source.android.com/#change,22722

Change-Id: I9bdd86971c94604477b91cdf47d6fac2c0bc186e
2011-06-14 16:59:19 -07:00
Shawn O. Pearce
0fc3a39829 Bump repo version to 1,10
Change-Id: Ifdc041e7152af31de413b9269f20000acd945b3b
2011-01-10 09:01:24 -08:00
Anthony Newnam
df14a70c45 Make path references OS independent
Change-Id: I5573995adfd52fd54bddc62d1d1ea78fb1328130
(cherry picked from commit b0f9a02394)

Conflicts:

	command.py
2011-01-09 17:39:19 -08:00
Shawn O. Pearce
f18cb76173 Encode the environment variables passed to git
Windows allows the environment to have unicode values.
This will cause Python to fail to execute the command.

Change-Id: I37d922c3d7ced0d5b4883f0220346ac42defc5e9
Signed-off-by: Shawn O. Pearce <sop@google.com>
2011-01-09 16:13:56 -08:00
Trond Norbye
d3fd537ea5 Exit with statuscode 0 for repo help init
The complete help text is printed, so the program executed successfully.

Some tools (like OpenGrok) detects the availibility of a program by
running it with a known set of options and check the return code.
It is an easy and portable way of checking for the existence of a program
instead of searching the path (and handle extensions) ourselves.

Change-Id: Ic13428c77be4a36d599ccb8c86d893308818eae3
2011-01-09 16:10:04 -08:00
Shawn O. Pearce
88443387b1 sync: Enable use of git clone --reference
Use git clone to initialize a new repository, and when possible
allow callers to use --reference to reuse an existing checkout as
the initial object storage area for the new checkout.

Change-Id: Ie27f760247f311ce484c6d3e85a90d94da2febfc
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-10-29 12:08:50 -07:00
Shawn O. Pearce
c8a300f639 Refactor Manifest to be XmlManifest
We'll soon be supporting two different manifest formats, but we
can't immediately remove support for the current XML one that is
in wide spread use within Android.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-05-29 09:31:28 -07:00
Shawn O. Pearce
fd89b67f5c Clarify options that control the repo executable version
Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-18 11:28:57 -07:00
Shawn O. Pearce
24d8dfbc34 Correct the REPO_URL in the wrapper script to android.git.kernel.org
Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-12-18 07:21:32 -08:00
Shawn O. Pearce
e284ad1d1a Add 'repo init --mirror' to download a complete forrest
The mirror option downloads a complete forrest (as described by the
manifest) and creates a replica of the remote repositories rather
than a client working directory.  This permits other clients to
sync off the mirror site.

A mirror can be positioned in a "DMZ", where the mirror executes
"repo sync" to obtain changes from the external upstream and
clients inside the protected zone operate off the mirror only,
and therefore do not require direct git:// access to the external
upstream repositories.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-11-05 18:08:32 -08:00
Shawn O. Pearce
45476c40c7 wrapper 1.6
Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-10-28 08:46:15 -07:00
Shawn O. Pearce
1619134720 Added missing wait after git-version call in wrapper
Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-10-28 08:44:18 -07:00
Shawn O. Pearce
7542d664de Remove the Python 2.4 dependency and use just 'python'
Many Linux distributions are including python2.5 by default, as
it is the latest stable release of the language.  Using python2.4
(and asking users to specifically install it) is just cruel and
unusual punishment.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-10-21 07:12:42 -07:00
The Android Open Source Project
cf31fe9b4f Initial Contribution 2008-10-21 07:00:00 -07:00