Commit Graph

876 Commits

Author SHA1 Message Date
Terence Haddock
4655e81a75 Add option to check status of projects in parallel.
Change-Id: I6ac653f88573def8bb3d96031d3570ff966251ad
2011-04-07 16:36:42 -04:00
Daniel Sandler
723c5dc3d6 Fix parallel sync on python < 2.6.
Event.isSet was renamed to is_set in 2.6, but we should
use the earlier syntax to avoid breaking compatibility
with older Python installations.

Change-Id: I41888ed38df278191d7496c1a6eed15e881733f4
2011-04-04 11:34:47 -04:00
Shawn O. Pearce
e6a0eeb80d sync: Fix syntax error on Python 2.4
Change-Id: I371d032d5a1ddde137721cbe2b24bfa38f20aaaa
Signed-off-by: Shawn O. Pearce <sop@google.com>
2011-03-22 19:04:47 -07:00
Victor Boivie
0960b5b53d Creating rr-cache
If git-rerere is enabled, it uses the rr-cache directory that
repo currently creates a symlink from, but doesn't create the
destination directory (inside the project's directory). Git
will then complain during merges and rebases.

This commit creates the rr-cache directory inside the project.

Change-Id: If8b57a04f022fc6ed6a7007d05aa2e876e6611ee
2011-03-17 09:19:51 -07:00
Doug Anderson
fc06ced9f9 Make 'repo sync -jN' exit with an error code in the case of sync errors.
The bug that this is fixing is described here:

http://code.google.com/p/chromium-os/issues/detail?id=6813

This fix allows the helper threads to signal the main thread that they
saw an error.  When the main thread sees the error, it will let all
existing threads finish, then exit with an error.

Change-Id: If3019bc6b0b3ab9304d49ed2eea53e9d57f3095a
2011-03-17 09:17:42 -07:00
Doug Anderson
fce89f218a Add 'list' command to repo.
This isn't a required command, but might be more discoverable for
repo newbies?

Change-Id: If357346f234774d42e04e024e65acdaf6dca6c62
2011-03-16 12:55:44 -07:00
Doug Anderson
37282b4b9c Support repo-level pre-upload hook and prep for future hooks.
All repo-level hooks are expected to live in a single project at the
top level of that project.  The name of the hooks project is provided
in the manifest.xml.  The manifest also lists which hooks are enabled
to make it obvious if a file somehow failed to sync down (or got
deleted).

Before running any hook, we will prompt the user to make sure that it
is OK.  A user can deny running the hook, allow once, or allow
"forever" (until hooks change).  This tries to keep with the git
spirit of not automatically running anything on the user's computer
that got synced down.  Note that individual repo commands can add
always options to avoid these prompts as they see fit (see below for
the 'upload' options).

When hooks are run, they are loaded into the current interpreter (the
one running repo) and their main() function is run.  This mechanism is
used (instead of using subprocess) to make it easier to expand to a
richer hook interface in the future.  During loading, the
interpreter's sys.path is updated to contain the directory containing
the hooks so that hooks can be split into multiple files.

The upload command has two options that control hook behavior:
  - no-verify=False, verify=False (DEFAULT):
    If stdout is a tty, can prompt about running upload hooks if needed.
    If user denies running hooks, the upload is cancelled.  If stdout is
    not a tty and we would need to prompt about upload hooks, upload is
    cancelled.
  - no-verify=False, verify=True:
    Always run upload hooks with no prompt.
  - no-verify=True, verify=False:
    Never run upload hooks, but upload anyway (AKA bypass hooks).
  - no-verify=True, verify=True:
    Invalid

Sample bit of manifest.xml code for enabling hooks (assumes you have a
project named 'hooks' where hooks are stored):
  <repo-hooks in-project="hooks" enabled-list="pre-upload" />

Sample main() function in pre-upload.py in hooks directory:
  def main(project_list, **kwargs):
    print ('These projects will be uploaded: %s' %
           ', '.join(project_list))
    print ('I am being a good boy and ignoring anything in kwargs\n'
           'that I don\'t understand.')
    print 'I fail 50% of the time.  How flaky.'
    if random.random() <= .5:
      raise Exception('Pre-upload hook failed.  Have a nice day.')

Change-Id: I5cefa2cd5865c72589263cf8e2f152a43c122f70
2011-03-11 11:53:23 -08:00
Skyler Kaufman
835cd6888f Post-nonexistent-revision crash sidestepped
Fix for the bug that leaves a fractional .git directory after attempting to
perform an initial sync to a nonexistent revision. Moved the initialization of
the working directory to after the revision ID has already been checked. Now,
no project/.git directory gets created at all if the revision ID is bad.

Change-Id: I0c9b2a59573410f1d11de7661591bf02e4ce326b
2011-03-08 13:48:24 -08:00
Doug Anderson
8ced8641c8 Renamed 'repo_hooks' function to '_ProjectHooks'.
This renaming was done for two reasons:
1. The hooks are actually project-level hooks, not repo-level
   hooks.  Since we are talking about adding repo-level hooks,
   It keeps things less confusing if we name the existing hooks
   to be "ProjectHooks"
2. The function is a private function in project.py and so
   should have capitalization to match.

I also added a docstring describing this function.

Change-Id: I1d30f5de08e8f9f99f78146e68c76f906782d97e
2011-02-01 09:57:29 -08:00
Doug Anderson
2536f80625 Fixed bug identifying 'commit-msg' files.
There was a minor typo that would cause repo to (I believe)
mistakenly identify any file that contained a substring of the
word 'commit-msg' as a commit message hook.  For example, the file
'mit' or the file 'msg' would be treated as a commit message hook.
I believe that it was intended that repo only recognize files
named exactly 'commit-msg'.

Change-Id: I93edbddf3da3cf0935641e6efb19b0a8ee6e2308
2011-02-01 09:53:56 -08:00
Shawn O. Pearce
e7a3bcbbb8 Merge branch 'stable'
* stable:
  Fix mirror clients with no worktree
2011-01-10 13:28:22 -08:00
Shawn O. Pearce
0ce6ca9c7b Fix mirror clients with no worktree
Commit "Make path references OS independent" (df14a70c45)
broke mirror clients by trying to invoke replace() on None
when there is no worktree.

Change-Id: Ie0a187058358f7dcdf83119e45cc65409c980f11
2011-01-10 13:26:34 -08:00
Shawn O. Pearce
25b51d8cb7 Merge branch 'stable'
* stable:
  Bump repo version to 1,10
2011-01-10 09:02:23 -08:00
Shawn O. Pearce
0fc3a39829 Bump repo version to 1,10
Change-Id: Ifdc041e7152af31de413b9269f20000acd945b3b
2011-01-10 09:01:24 -08:00
Shawn O. Pearce
cef005c3e8 Merge branch 'maint'
* maint:
  help: Don't show empty Summary or Description sections
  sync: Run `git gc --auto` after fetch
  Add "repo branch" as an alias for "repo branches"
  upload: Catch and cleanly report connectivity errors
  forall: Silently skip missing projects
  Fix to display the usage message of the command download when the user don't provide any arguments to 'repo download'.
  Use os.environ.copy() instead of dict()
  Make path references OS independent
2011-01-09 17:39:30 -08:00
Shawn O. Pearce
c7c57e34db help: Don't show empty Summary or Description sections
Signed-off-by: Shawn O. Pearce <sop@google.com>
(cherry picked from commit 60e679209a)
2011-01-09 17:39:22 -08:00
Shawn O. Pearce
0d2b61f11d sync: Run git gc --auto after fetch
Users may wind up with a lot of loose object content in projects they
don't frequently make changes in, but that are modified by others.

Since we bypass many git code paths that would have otherwise called
out to `git gc --auto`, its possible for these projects to have
their loose object database grow out of control.  To help prevent
that, we now invoke it ourselves during the network half of sync.

Signed-off-by: Shawn O. Pearce <sop@google.com>
(cherry picked from commit 1875ddd47c)
2011-01-09 17:39:22 -08:00
Mike Lockwood
2bf9db0d3b Add "repo branch" as an alias for "repo branches"
For those of us that are used to typing "git branch".

Signed-off-by: Mike Lockwood <lockwood@android.com>
(cherry picked from commit 33f0e786bb)
2011-01-09 17:39:22 -08:00
Shawn O. Pearce
f00e0ce556 upload: Catch and cleanly report connectivity errors
Instead of giving a Python backtrace when there is a connectivity
problem during repo upload, report that we cannot access the host,
and why, with a halfway decent error message.

Bug: REPO-45
Change-Id: I9a45b387e86e48073a2d99bd6d594c1a7d6d99d4
Signed-off-by: Shawn O. Pearce <sop@google.com>
(cherry picked from commit d2dfac81ad)
2011-01-09 17:39:22 -08:00
Shawn O. Pearce
1b5a4a0c5d forall: Silently skip missing projects
If a project is missing locally, it might be OK to skip over it
and continue running the same command in other projects.

Bug: REPO-43
Change-Id: I64f97eb315f379ab2c51fc53d24ed340b3d09250
Signed-off-by: Shawn O. Pearce <sop@google.com>
(cherry picked from commit d4cd69bdef)
2011-01-09 17:39:22 -08:00
Thiago Farina
de8b2c4276 Fix to display the usage message of the command download when the user
don't provide any arguments to 'repo download'.

Signed-off-by: Thiago Farina <thiago.farina@gmail.com>
(cherry picked from commit 840ed0fab7)
2011-01-09 17:39:22 -08:00
Shawn O. Pearce
727ee98a40 Use os.environ.copy() instead of dict()
Signed-off-by: Shawn O. Pearce <sop@google.com>
(cherry picked from commit 3218c13205)
2011-01-09 17:39:22 -08:00
Anthony Newnam
df14a70c45 Make path references OS independent
Change-Id: I5573995adfd52fd54bddc62d1d1ea78fb1328130
(cherry picked from commit b0f9a02394)

Conflicts:

	command.py
2011-01-09 17:39:19 -08:00
Shawn O. Pearce
71cab95b4c Merge branch 'stable'
* stable:
  Encode the environment variables passed to git
  Exit with statuscode 0 for repo help init
2011-01-09 17:29:50 -08:00
Shawn O. Pearce
f18cb76173 Encode the environment variables passed to git
Windows allows the environment to have unicode values.
This will cause Python to fail to execute the command.

Change-Id: I37d922c3d7ced0d5b4883f0220346ac42defc5e9
Signed-off-by: Shawn O. Pearce <sop@google.com>
2011-01-09 16:13:56 -08:00
Trond Norbye
d3fd537ea5 Exit with statuscode 0 for repo help init
The complete help text is printed, so the program executed successfully.

Some tools (like OpenGrok) detects the availibility of a program by
running it with a known set of options and check the return code.
It is an easy and portable way of checking for the existence of a program
instead of searching the path (and handle extensions) ourselves.

Change-Id: Ic13428c77be4a36d599ccb8c86d893308818eae3
2011-01-09 16:10:04 -08:00
Shawn O. Pearce
9275fd4329 Merge branch 'stable'
* stable:
  Fixed race condition in 'repo sync -jN' that would open multiple masters.
2010-12-22 14:46:15 -08:00
Doug Anderson
0048b69c03 Fixed race condition in 'repo sync -jN' that would open multiple masters.
This fixes the SSH Control Masters to be managed in a thread-safe
fashion.  This is important because "repo sync -jN" uses threads to
sync more than one repository at the same time.  The problem didn't
show up earlier because it was masked if all of the threads tried to
connect to the same host that was used on the "repo init" line.
2010-12-21 13:39:23 -08:00
Shawn O. Pearce
13f3da50d4 Merge branch 'stable'
* stable: (33 commits)
  Added feature to print a <notice> from manifest at the end of a sync.
  sync: Use --force-broken to continue other projects
  upload: Remove --replace option
  sync --quiet: be more quiet
  sync: Enable use of git clone --reference
  Only delete corrupt pickle config files if they exist
  Don't allow git fetch to start ControlMaster
  Check for existing SSH ControlMaster
  Fix for handling values of EDITOR which contain a space.
  upload: Fix --replace flag
  rebase: Pass through more options
  upload: Allow review.HOST.username to override email
  upload -t: Automatically include local branch name
  Warn users before uploading if there are local changes
  sync: Try fetching a tag as a last resort before giving up
  rebase: Automatically rebase branch on upstrea
  upload: Automatically --cc folks in review.URL.autocopy
  Fix format string bugs in grep
  Do not invoke ssh with -p argument when no port has been specified.
  Allow files to be copied into new folders
  ...

Conflicts:
	git_config.py
	manifest_xml.py
	subcmds/init.py
	subcmds/sync.py
	subcmds/upload.py

Change-Id: I4756a6908277e91505c35287a122a775b68f4df5
2010-12-07 11:13:29 -08:00
Shawn O. Pearce
3218c13205 Use os.environ.copy() instead of dict()
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-12-07 08:46:14 -08:00
Anthony Newnam
b0f9a02394 Make path references OS independent
Change-Id: I5573995adfd52fd54bddc62d1d1ea78fb1328130
2010-11-29 13:17:53 -06:00
Doug Anderson
2b8db3ce3e Added feature to print a <notice> from manifest at the end of a sync.
This feature is used to convey information on a when a branch has
ceased development or if it is an experimental branch with a few
gotchas, etc.

You add it to your manifest XML by doing something like this:
<manifest>
  <notice>
    NOTE TO DEVELOPERS:
      If you checkin code, you have to pinky-swear that it contains no bugs.
      Anyone who breaks their promise will have tomatoes thrown at them in the
      team meeting.  Be sure to bring an extra set of clothes.
  </notice>

  <remote ... />
  ...
</manifest>

Carriage returns and indentation are relevant for the text in this tag.

This feature was requested by Anush Elangovan on the ChromiumOS team.
2010-11-01 15:08:06 -07:00
Andrei Warkentin
5df6de075e sync: Use --force-broken to continue other projects
This adds a new flag -f/--force-broken that will allow the rest of
the sync process to continue instead of bailing when a particular
project fails to sync.

Change-Id: I23680f2ee7927410f7ed930b1d469424c9aa246e
Signed-off-by: Andrei Warkentin <andreiw@motorola.com>
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-10-29 12:20:01 -07:00
Ficus Kirkpatrick
a0de6e8eab upload: Remove --replace option
It hasn't been necessary for a long time, and its
functionality can be accomplished with 'git push'.

Change-Id: Ic00d3adbe4cee7be3955117489c69d6e90106559
2010-10-29 12:12:56 -07:00
Shawn O. Pearce
16614f86b3 sync --quiet: be more quiet
Change-Id: I5e8363c7b32e4546d1236cfc5a32e01c3e5ea8e6
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-10-29 12:08:57 -07:00
Shawn O. Pearce
88443387b1 sync: Enable use of git clone --reference
Use git clone to initialize a new repository, and when possible
allow callers to use --reference to reuse an existing checkout as
the initial object storage area for the new checkout.

Change-Id: Ie27f760247f311ce484c6d3e85a90d94da2febfc
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-10-29 12:08:50 -07:00
Ulrik Sjölin
99482ae58a Only delete corrupt pickle config files if they exist
os.remove() raises OSError if the file being removed doesn't exist.
Check before calling to ensure we don't raise a useless exception
on an already deleted file.

Change-Id: I44c1c7dd97a47fcab8afb6c18fdf179158b6dab7
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-10-29 08:25:04 -07:00
Doug Anderson
ec1df9b7f6 Don't allow git fetch to start ControlMaster
To avoid connectivity problems, we don't want the ssh process
that is started by git fetch to become a ControlMaster for the
overall sync task.  If it did, we would lose connectivity when
git fetch was finished with the current project, causing later
projects to not fetch efficiently.

Change-Id: I8d0dcf9b361276ff8c8b5a6324cbd4a501e9c4dd
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-10-29 08:15:14 -07:00
Doug Anderson
06d029c1c8 Check for existing SSH ControlMaster
Be more thorough about checking for an existing ssh master by
running a test command first, and only opening up a new master
if the test fails to connect.

Change-Id: I56fe8e7b4dbc123675b7f259e81d359ed0cd55cf
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-10-29 08:14:56 -07:00
Patrick Dubroy
b715b14807 Fix for handling values of EDITOR which contain a space.
The shell swallows the 0th arg, which was the filename. Simple fix
is to pass in an extra arg for the shell to swallow.

Change-Id: Iad6304ba9ccea6e7262ee06ef87d3dac57dbde81
2010-08-06 17:05:04 -07:00
Shawn O. Pearce
60829ba72f upload: Fix --replace flag
--replace started to fail due to a Python error, I forgot to pass
through the opt structure to the replace function.

Change-Id: Ifcd7a0c715c3fd9070a4c58208612a626382de35
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-07-16 07:42:45 -07:00
Shawn O. Pearce
a22f99ae41 rebase: Pass through more options
Passing through --whitespace=fix to rebase can be useful
to clean up a branch prior to uploading it for review.

Change-Id: Id85f1912e5e11ff9602e3b342c2fd7441abe67d7
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-07-15 17:43:02 -07:00
Shawn O. Pearce
3575b8f8bd upload: Allow review.HOST.username to override email
Some users might need to use a different login name than the local
part of their email address for their Gerrit Code Review user
account.  Allow it to be overridden with the review.HOST.username
configuration variable.

Change-Id: I714469142ac7feadf09fee9c26680c0e09076b75
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-07-15 17:03:19 -07:00
Shawn O. Pearce
a5ece0e050 upload -t: Automatically include local branch name
If the -t flag is given to upload, the local branch name is
automatically sent to Gerrit Code Review as the topic branch name
for the change(s).  This requires the server to be Gerrit Code
Review v2.1.3-53-gd50c94e or later, which isn't widely deployed
right now, so the default is opt-out.

Change-Id: I034fcacb405b7cb909147152db427fe69dd7bcbf
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-07-15 16:52:42 -07:00
Anthony Newnam
cc50bac8c7 Warn users before uploading if there are local changes
Change-Id: I231d7b6a3211e9f5ec71a542a0109b0c195d5e40
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-07-15 16:43:58 -07:00
Julius Gustavsson
0cb1b3f687 sync: Try fetching a tag as a last resort before giving up
If a tagged commit is not reachable by the fetch refspec configured
for the git (usually refs/heads/*) it will not be downloaded by
'git fetch'.  The tag can however be downloaded with 'git fetch
--tags' or 'git fetch tag <tag>'.

This patch fixes the situation when a tag is not found after a
'git fetch'. Repo will issue 'git fetch tag <tag>' before giving
up completely.

Change-Id: I87796a5e1d51fcf398f346a274b7a069df37599a
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-07-15 16:38:08 -07:00
Daniel Sandler
9e426aa432 rebase: Automatically rebase branch on upstrea
Usage: repo rebase [[-i] <project>...]

Rebases the current topic branch of the specified (or all)
projects against the appropriate upstream.

Note: Interactive rebase is currently only supported when
exactly one project is specified on the command line.

Change-Id: I7376e35f27a6585149def82938c1ca99f36db2c4
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-07-15 16:35:31 -07:00
Ben Komalo
08a3f68d38 upload: Automatically --cc folks in review.URL.autocopy
The upload command will read review.URL.autocopy from the project's
configuration and append the list of e-mails specified to the
--cc argument of the upload command if a non-empty --re argument
was provided.

Change-Id: I2424517d17dd3444b20f0e6a003be6e70b8904f6
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-07-15 16:30:32 -07:00
Sebastian Schmidt
feb39d61ef Fix format string bugs in grep
This fixes some format string bugs in grep which cause repo to with
"TypeError: not enough arguments for format string" when grepping and
the output contains a valid Python format string.

Change-Id: Ice8968ea106148d409490e4f71a2833b0cc80816
2010-06-17 19:09:37 -07:00
Josh Guilfoyle
7198572dd7 Do not invoke ssh with -p argument when no port has been specified.
This change allows local SSH configuration to choose the port number
to use when not explicitly set in the manifest.

(cherry picked from commit 4c0f670465)

Change-Id: Ibea99cfe46b6a2cc27f754cc3944a2fe10f6fda4
2010-06-08 11:08:11 -07:00