Commit Graph

86 Commits

Author SHA1 Message Date
Victor Boivie
08c880db18 Smart tag support
This is an evolution of 'smart-sync' that adds a new option, -t,
that allows you to specify a tag/label to use instead of the
"latest good build" on the current manifest branch which -s does.

Signed-off-by: Victor Boivie <victor.boivie@sonyericsson.com>
Change-Id: I8c20fd91104a6aafa0271d4d33f6c4850aade17e
2011-07-20 07:13:48 -07:00
Daniel Sandler
723c5dc3d6 Fix parallel sync on python < 2.6.
Event.isSet was renamed to is_set in 2.6, but we should
use the earlier syntax to avoid breaking compatibility
with older Python installations.

Change-Id: I41888ed38df278191d7496c1a6eed15e881733f4
2011-04-04 11:34:47 -04:00
Shawn O. Pearce
e6a0eeb80d sync: Fix syntax error on Python 2.4
Change-Id: I371d032d5a1ddde137721cbe2b24bfa38f20aaaa
Signed-off-by: Shawn O. Pearce <sop@google.com>
2011-03-22 19:04:47 -07:00
Doug Anderson
fc06ced9f9 Make 'repo sync -jN' exit with an error code in the case of sync errors.
The bug that this is fixing is described here:

http://code.google.com/p/chromium-os/issues/detail?id=6813

This fix allows the helper threads to signal the main thread that they
saw an error.  When the main thread sees the error, it will let all
existing threads finish, then exit with an error.

Change-Id: If3019bc6b0b3ab9304d49ed2eea53e9d57f3095a
2011-03-17 09:17:42 -07:00
Shawn O. Pearce
0d2b61f11d sync: Run git gc --auto after fetch
Users may wind up with a lot of loose object content in projects they
don't frequently make changes in, but that are modified by others.

Since we bypass many git code paths that would have otherwise called
out to `git gc --auto`, its possible for these projects to have
their loose object database grow out of control.  To help prevent
that, we now invoke it ourselves during the network half of sync.

Signed-off-by: Shawn O. Pearce <sop@google.com>
(cherry picked from commit 1875ddd47c)
2011-01-09 17:39:22 -08:00
Shawn O. Pearce
f18cb76173 Encode the environment variables passed to git
Windows allows the environment to have unicode values.
This will cause Python to fail to execute the command.

Change-Id: I37d922c3d7ced0d5b4883f0220346ac42defc5e9
Signed-off-by: Shawn O. Pearce <sop@google.com>
2011-01-09 16:13:56 -08:00
Doug Anderson
2b8db3ce3e Added feature to print a <notice> from manifest at the end of a sync.
This feature is used to convey information on a when a branch has
ceased development or if it is an experimental branch with a few
gotchas, etc.

You add it to your manifest XML by doing something like this:
<manifest>
  <notice>
    NOTE TO DEVELOPERS:
      If you checkin code, you have to pinky-swear that it contains no bugs.
      Anyone who breaks their promise will have tomatoes thrown at them in the
      team meeting.  Be sure to bring an extra set of clothes.
  </notice>

  <remote ... />
  ...
</manifest>

Carriage returns and indentation are relevant for the text in this tag.

This feature was requested by Anush Elangovan on the ChromiumOS team.
2010-11-01 15:08:06 -07:00
Andrei Warkentin
5df6de075e sync: Use --force-broken to continue other projects
This adds a new flag -f/--force-broken that will allow the rest of
the sync process to continue instead of bailing when a particular
project fails to sync.

Change-Id: I23680f2ee7927410f7ed930b1d469424c9aa246e
Signed-off-by: Andrei Warkentin <andreiw@motorola.com>
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-10-29 12:20:01 -07:00
Shawn O. Pearce
16614f86b3 sync --quiet: be more quiet
Change-Id: I5e8363c7b32e4546d1236cfc5a32e01c3e5ea8e6
Signed-off-by: Shawn O. Pearce <sop@google.com>
2010-10-29 12:08:57 -07:00
Roy Lee
18afd7f679 sync: support --jobs to fetch projects simultaneously
This patch does two things for being compatibile with
those Python which are built without threading support:

1. As the Python document and Shawn suggested, import dummy_threading
   when the threading is not available.

2. Reserve the single threaded code and make it default.
   In cases the --jobs does not work properly with dummy_threading,
   we still have a safe fallback.

Change-Id: I40909ef8e9b5c22f315c0a1da9be38eed8b0a2dc
2010-05-27 14:54:20 -07:00
Nico Sallembien
6623b21e10 Aliasing sync -s to 'smartsync'
This alias will let people use this command without having to
remember the option.

Change-Id: I3256d9e8e884c5be9e77f70e9cfb73e0f0c544c6
2010-05-17 09:58:55 -07:00
Nico Sallembien
719965af35 Override manifest file only after it is fully written to disk.
We called "Override()" before closing the file passed in argument.

Change-Id: I15adb99deb14297ef72fcb1b0945eb246f172fb0
2010-04-26 11:20:22 -07:00
Nico Sallembien
5732e47ebb Strip refs/heads in the branch sent to the manifest server.
The manifest server doesn't want to have refs/heads passed to it, so
we need to strip that when the branch contains it.

Change-Id: I044f8a9629220e886fd5e02e3c1ac4b4bb6020ba
2010-04-26 11:19:07 -07:00
Anthony
f3fdf823cf sync: Safely skip already deleted projects
Do not error if a project is missing on the filesystem, is deleted
from manifest.xml, but still exists in project.list.

Change-Id: I1d13e435473c83091e27e4df571504ef493282dd
2010-04-14 14:21:50 -07:00
Nico Sallembien
a1bfd2cd72 Add a 'smart sync' option to repo sync
This option allows the user to specify a manifest server to use when
syncing. This manifest server will provide a manifest pegging each
project to a known green build. This allows developers to work on a
known good tree that is known to build and pass tests, preventing
failed builds to hamper productivity.

The manifest used is not "sticky" so as to allow subsequent
'repo sync' calls to sync to the tip of the tree.

Change-Id: Id0a24ece20f5a88034ad364b416a1dd2e394226d
2010-04-13 10:20:37 -07:00
Nico Sallembien
9bb1816bdc Fixing project renaming bug.
This bug happens when a project gets added to the manifest, and
then is renamed. Users who happened to have run "repo sync" after
the project was added but before the rename happened will try to
read the data from the old project, as the manifest was only updated
after all projects were updated successfully.
2009-12-10 15:24:45 -08:00
Shawn O. Pearce
9fb29ce123 sync: Keep the project.list file sorted
Its easier to locate an entry visually if the file is sorted.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-06-04 20:41:26 -07:00
Shawn O. Pearce
3a68bb4c7f sync: Tolerate blank lines in project.list
If a line is blank in project.list, its not a relevant project path,
so skip over it.  Existing project.list files may have blank lines if
sync was run with no projects at all, and the file was created empty.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-06-04 16:21:01 -07:00
Shawn O. Pearce
cd1d7ff81e sync: Don't process project.list in a mirror
We have no working tree, so we cannot update the project.list
state file, nor should we try to delete a directory if a project is
removed from the manifest.  Clients would still need the repository
for historical records.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-06-04 16:20:02 -07:00
Jaikumar Ganesh
8135cdc53c Delete empty parent subdirs after deleting obsolete paths.
After sync, we delete obsolete project paths.
Iterate and delete parent subdirs which are empty.
Tested on projects within subdirectories.
2009-06-02 15:08:45 -07:00
Jaikumar Ganesh
4f2517ff11 Update project paths after sync.
After a repo sync, some of the project paths might need
to be removed. This changes maintains a list of project
paths from the previous sync operation and compares.
2009-06-02 11:00:53 -07:00
Shawn O. Pearce
3c8dea1f8d Change project.revision to revisionExpr and revisionId
The revisionExpr field now holds an expression from the manifest,
such as "refs/heads/master", while revisionId holds the current
commit-ish SHA-1 of the revisionExpr.  Currently that is only
filled in if the manifest points directly to a SHA-1.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-05-29 18:45:20 -07:00
Shawn O. Pearce
eb7af87bcf Document the SSH ControlMaster behavior of repo sync
Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-21 08:28:06 -07:00
Shawn O. Pearce
fd89b67f5c Clarify options that control the repo executable version
Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-18 11:28:57 -07:00
Shawn O. Pearce
f690687671 Only fetch repo once-per-day under normal 'repo sync' usage
Its unlikely that a new version of repo will be delivered in any
given day, so we now check only once every 24 hours to see if repo
has been updated.  This reduces the sync cost, as we no longer need
to contact the repo distribution servers every time we do a sync.

repo selfupdate can still be used to force a check.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-18 10:49:00 -07:00
Shawn O. Pearce
350cde4c4b Change repo sync to be more friendly when updating the tree
We now try to sync all projects that can be done safely first, before
we start rebasing user commits over the upstream.  This has the nice
effect of making the local tree as close to the upstream as possible
before the user has to start resolving merge conflicts, as that extra
information in other projects may aid in the conflict resolution.

Informational output is buffered and delayed until calculation for
all projects has been done, so that the user gets one concise list
of notice messages, rather than it interrupting the progress meter.

Fast-forward output is now prefixed with the project header, so the
user can see which project that update is taking place in, and make
some relation of the diffstat back to the project name.

Rebase output is now prefixed with the project header, so that if
the rebase fails, the user can see which project we were operating
on and can try to address the failure themselves.

Since rebase sits on a detached HEAD, we now look for an in-progress
rebase during sync, so we can alert the user that the given project
is in a state we cannot handle.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-16 11:21:18 -07:00
Shawn O. Pearce
e756c412e3 Add 'repo selfupdate' to upgrade only repo
Users may want to upgrade only repo to the latest release, but
leave their working tree state alone and avoid 'repo sync'.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-13 11:53:53 -07:00
Shawn O. Pearce
68194f42b0 Add a project progress meter to 'repo sync'
This way users can see how much is left during fetch.  Its
especially useful when most syncs are no-ops but there are
hundreds of repositories to poll.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-10 19:01:04 -07:00
Shawn O. Pearce
b1562faee0 Add 'repo sync -l' to only do local operations
This permits usage of 'repo sync' while offline, as we bypass the
network based portions of the code and do only the local sync.

An example use case might be:

  repo sync -n  ; # while we have network
  ... some time later ...
  repo sync -l  ; # while without network, come up to date

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-10 17:08:02 -07:00
Shawn O. Pearce
3e768c9dc7 Add 'repo sync -d' to detach projects from their current topic
The -d flag moves the project back to a detached HEAD state,
matching what is listed in the manifest.  This can be useful to
set a client to something stable (or at least well-known), such as
before a sequence of 'repo download' commands are used to get some
changes for testing.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-10 17:08:02 -07:00
Shawn O. Pearce
96fdcef9e3 Add 'repo sync -n' to only do the network transfer
This makes it easier to update all repositories, without actually
impacting the working directory, or learning about how to use
`repo forall -c 'git fetch $REPO_REMOTE' `.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-10 17:07:52 -07:00
Shawn O. Pearce
2a1ccb2b0c Hide the internal sync --repo-upgraded flag from users
This is only meant to be passed through while repo upgrades itself
during a sync.  It should never be something a user invokes on
their own.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-04-10 17:07:32 -07:00
Shawn O. Pearce
c95583bf4f Don't permit users to run repo status in a mirror client
If a client was created with "repo init --mirror" then there are
no working directories present, and no files checked out.  Using
a command like "repo status" in this context makes no sense, and
actually throws back a Pytyon traceback at the console when the
underlying commands fail out.

We now tag commands with the MirrorSafeCommand type if they are
able to be executed within a mirror directory safely.  Using a
command in a mirror which lacks this base class results in a
useful error letting you know the command isn't supported.

Bug: REPO-14
Signed-off-by: Shawn O. Pearce <sop@google.com>
2009-03-03 17:47:06 -08:00
Shawn O. Pearce
e284ad1d1a Add 'repo init --mirror' to download a complete forrest
The mirror option downloads a complete forrest (as described by the
manifest) and creates a replica of the remote repositories rather
than a client working directory.  This permits other clients to
sync off the mirror site.

A mirror can be positioned in a "DMZ", where the mirror executes
"repo sync" to obtain changes from the external upstream and
clients inside the protected zone operate off the mirror only,
and therefore do not require direct git:// access to the external
upstream repositories.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-11-05 18:08:32 -08:00
Shawn O. Pearce
c9ef744c7b Install a default pre-auto-gc hook in all repositories
This hook is evaluated by `git gc --auto` to determine if it is a
good idea to execute a GC at this time, or defer it to some later
date.  When working on a laptop its a good idea to avoid GC if you
are on battery power as the extra CPU and disk IO would consume a
decent amount of the charge.

The hook is the standard sample hook from git.git contrib/hooks,
last modified in git.git by 84ed4c5d117d72f02cc918e413b9861a9d2846d7.
I added the GPLv2 header to the script to ensure the license notice
is clear, as it does not match repo's own APLv2 license.

We only update hooks during initial repository creation or on
a repo sync.  This way we don't incur huge overheads from the
hook stat operations during "repo status" or even the normal
"repo sync" cases.

Signed-off-by: Shawn O. Pearce <sop@google.com>
2008-11-03 11:00:44 -08:00
The Android Open Source Project
cf31fe9b4f Initial Contribution 2008-10-21 07:00:00 -07:00