Compare commits

..

11 Commits

Author SHA1 Message Date
34bc5712eb README: add install details
Change-Id: I57043449a7927068fa5735cb71633353e1039532
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/245816
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-11-18 19:17:48 +00:00
70c54dc255 upload/editor: fix bytes/string confusion
The upload module tries to turn the strings into bytes before passing
to EditString, but it combines bytes & strings causing an error.  The
return value might be bytes or string, but the caller only expects a
string.  Lets simplify this by sticking to strings everywhere and have
EditString take care of converting to/from bytes when reading/writing
the underlying files.  This also avoids possible locale confusion when
reading the file by forcing UTF-8 everywhere.

Bug: https://crbug.com/gerrit/11929
Change-Id: I07b146170c5e8b5b0500a2c79e4213cd12140a96
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/245621
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-11-16 23:55:30 +00:00
6da17751ca prune: handle branches that track missing branches
Series of steps:
* Create a local "b1" branch with `repo start b1` that tracks a remote
  branch (totally fine)
* Manually create a local "b2" branch with `git branch --track b1 b2`
  that tracks the local "b1" (uh-oh...)
* Delete the local "b1" branch manually or via `repo prune` (....)
* Try to process the "b2" branch with `repo prune`

Since b2 tracks a branch that no longer exists, everything blows up
at this point as we try to probe the non-existent ref.  Instead, we
should flag this as unknown and leave it up to the user to resolve.

This probably could come up if a local branch was tracking a remote
branch that was deleted from the server, and users ran something like
`repo sync --prune` which cleaned up the remote refs.

Bug: https://crbug.com/gerrit/11485
Change-Id: I6b6b6041943944b8efa6e2ad0b8b10f13a75a5c2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/236793
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Kirtika Ruchandani <kirtika@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
2019-11-16 19:55:02 +00:00
2ba5a1e963 sync: try to fast forward merge branches before checking published state
If the local branch changed state since its last upload, the data
cached in .git/config related to the last uploaded CL might not be
that relevant.  If we're able to fast forward merge to the latest
tree state, then let's do that.  This would be akin to checking
out a detached head before syncing where we already switch state.

If we aren't able to fast forward merge, then it's not a big deal
as we'll continue on to the existing branch checking logic.

This would be easy to reproduce by doing something like:
  $ repo start foo .
  $ git revert HEAD
  $ repo upload --cbr .
  $ git reset --hard HEAD^
  <CL is merged>
  $ repo sync .
  <we can fast forward>

Change-Id: I7d62f3d1ba5314a349d85b4dbb0ec8352eca18bb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/238552
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
2019-11-13 20:29:33 +00:00
3538dd224d sync: merge project updates with status bar
The current sync output displays "Fetching project" and "Checking out
project" messages and progress bar updates independently leading to a
lot of spam.  Lets merge these periodic outputs with the status bar to
get a little bit tighter output in the normal case.  This doesn't solve
all our problems, but gets us closer.

Bug: https://crbug.com/gerrit/11293
Change-Id: Icd627830af4dd934a9355b7ace754b56dc96cfef
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244934
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-11-12 23:33:51 +00:00
b610b850ac sync: add sanity check for local checkouts missing network
If you run `repo sync -l foo` without first `repo sync -n foo`,
repo sets up an invalid gitdir tree that gets wedged and requires
manual recovery.  Add a sanity check to abort cleanly first.

Change-Id: Iad865ea860a3f1fd2f39ce683fe66bd4380745a5
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244732
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-11-12 23:14:28 +00:00
dff919493a sync: report list of failing git trees
When repo sync fails because some git trees are not in clean state and
as such can not be rebased automatically, it is a pain to figure out
which trees are the culprits.

With this patch the list of offending trees is printed when repo sync
reports checkout errors.

TEST=ran 'repo sync' and observed the proper list of directories show
     up after the final error message

Bug: https://crbug.com/gerrit/11293
Change-Id: Icdf1a03e9014ecb184f331f513cc9a2efc7d11ed
Signed-off-by: Vadim Bendebury <vbendeb@google.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244053
Reviewed-by: Mike Frysinger <vapier@google.com>
2019-11-12 21:08:54 +00:00
3164d40e22 use open context managers in more places
Use open() as a context manager to simplify the close logic and make
the code easier to read & understand.  This is also more Pythonic.

Change-Id: I579d03cca86f99b2c6c6a1f557f6e5704e2515a7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244734
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-11-12 03:44:39 +00:00
f454512619 sync: make .git init more robust
Hitting Ctrl-C in the middle of this func will leave the .git in a
bad state that requires manual recovery.  The code tries to catch
all exceptions and recover by deleting the incomplete .git dir, but
it omits KeyboardInterrupt which Exception misses.

We could add that to the recovery path, but we can make this more
robust with a different approach: set up everything in .git.tmp/
and only move it to .git/ once we've fully initialized it.

Change-Id: I0f5b97f2e19fc39cffc3e5e23993a2da7220f4e3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244733
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-11-12 03:44:33 +00:00
b466854bed forall: add an --ignore-missing option
In CrOS, our infra has to deal with partial checkouts constantly
(for a variety of reasons).  To help reset back to a good state,
we run git commands via `repo forall`, but don't care about the
missing checkouts.  Add a flag so we can disambiguate between
missing repos and failing git subcommands.

Bug: https://crbug.com/1013377
Bug: https://crbug.com/1013623
Change-Id: Ie3498c6d111276c60d2ecedbba21bfa778588d50
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/241935
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-10-22 14:10:34 +00:00
d1e93dd58a python-support: adjust major versions
The plan previously documented was <=1.13.x is Python 2 and >=1.14.x
is Python 3.  Other projects that migrated Python versions and drop
support for older have tended to take a more drastic version jump to
make it clearer to users.  So lets adjust the plan to say <=1.x will
support Python 2, and >=2.x will be Python 3-only.

This also allows us to harmonize the repo launcher version.  It is
currently sitting at v1.26 and has been incremented independently of
the repo version for the life of the project.  While we might know
these lower nuances, pretty much no one else does and it just leads
to confusion: do I know version 1.26 or version 1.13.7?  Or do I
have both?  What does that even mean?

Once we update the major version to 2.0.0, we can also adjust the
launcher script to 2.0.0, and then the launcher release process will
be tied to a new repo release in general.

Bug: https://crbug.com/gerrit/10418
Change-Id: Idb2257371a06e56d2923cf717345c028f49176a2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/240372
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
2019-10-11 06:56:03 +00:00
16 changed files with 306 additions and 141 deletions

View File

@ -14,3 +14,22 @@ that you can put anywhere in your path.
* [repo Manifest Format](./docs/manifest-format.md)
* [repo Hooks](./docs/repo-hooks.md)
* [Submitting patches](./SUBMITTING_PATCHES.md)
## Install
Many distros include repo, so you might be able to install from there.
```sh
# Debian/Ubuntu.
$ sudo apt-get install repo
# Gentoo.
$ sudo emerge dev-vcs/repo
```
You can install it manually as well as it's a single script.
```sh
$ mkdir -p ~/.bin
$ PATH="${HOME}/.bin:${PATH}"
$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/.bin/repo
$ chmod a+rx ~/.bin/repo
```

View File

@ -7,9 +7,9 @@ their old LTS/corp systems and have little power to change the system.
## Summary
* Python 3.6 (released Dec 2016) is required by default starting with repo-1.14.
* Python 3.6 (released Dec 2016) is required by default starting with repo-2.x.
* Older versions of Python (e.g. v2.7) may use the legacy feature-frozen branch
based on repo-1.13.
based on repo-1.x.
## Overview

View File

@ -68,11 +68,14 @@ least one of these before using this command.""", file=sys.stderr)
def EditString(cls, data):
"""Opens an editor to edit the given content.
Args:
data : the text to edit
Args:
data: The text to edit.
Returns:
new value of edited text; None if editing did not succeed
Returns:
New value of edited text.
Raises:
EditorError: The editor failed to run.
"""
editor = cls._GetEditor()
if editor == ':':
@ -80,7 +83,7 @@ least one of these before using this command.""", file=sys.stderr)
fd, path = tempfile.mkstemp()
try:
os.write(fd, data)
os.write(fd, data.encode('utf-8'))
os.close(fd)
fd = None
@ -106,11 +109,8 @@ least one of these before using this command.""", file=sys.stderr)
raise EditorError('editor failed with exit status %d: %s %s'
% (rc, editor, path))
fd2 = open(path)
try:
return fd2.read()
finally:
fd2.close()
with open(path, mode='rb') as fd2:
return fd2.read().decode('utf-8')
finally:
if fd:
os.close(fd)

View File

@ -276,22 +276,16 @@ class GitConfig(object):
return None
try:
Trace(': parsing %s', self.file)
fd = open(self._json)
try:
with open(self._json) as fd:
return json.load(fd)
finally:
fd.close()
except (IOError, ValueError):
platform_utils.remove(self._json)
return None
def _SaveJson(self, cache):
try:
fd = open(self._json, 'w')
try:
with open(self._json, 'w') as fd:
json.dump(cache, fd, indent=2)
finally:
fd.close()
except (IOError, TypeError):
if os.path.exists(self._json):
platform_utils.remove(self._json)
@ -773,15 +767,12 @@ class Branch(object):
self._Set('merge', self.merge)
else:
fd = open(self._config.file, 'a')
try:
with open(self._config.file, 'a') as fd:
fd.write('[branch "%s"]\n' % self.name)
if self.remote:
fd.write('\tremote = %s\n' % self.remote.name)
if self.merge:
fd.write('\tmerge = %s\n' % self.merge)
finally:
fd.close()
def _Set(self, key, value):
key = 'branch.%s.%s' % (self.name, key)

View File

@ -141,18 +141,11 @@ class GitRefs(object):
def _ReadLoose1(self, path, name):
try:
fd = open(path)
except IOError:
return
try:
try:
with open(path) as fd:
mtime = os.path.getmtime(path)
ref_id = fd.readline()
except (IOError, OSError):
return
finally:
fd.close()
except (IOError, OSError):
return
try:
ref_id = ref_id.decode()

View File

@ -241,14 +241,15 @@ def _makelongpath(path):
return path
def rmtree(path):
def rmtree(path, ignore_errors=False):
"""shutil.rmtree(path) wrapper with support for long paths on Windows.
Availability: Unix, Windows."""
onerror = None
if isWindows():
shutil.rmtree(_makelongpath(path), onerror=handle_rmtree_error)
else:
shutil.rmtree(path)
path = _makelongpath(path)
onerror = handle_rmtree_error
shutil.rmtree(path, ignore_errors=ignore_errors, onerror=onerror)
def handle_rmtree_error(function, path, excinfo):

View File

@ -39,7 +39,7 @@ class Progress(object):
self._print_newline = print_newline
self._always_print_percentage = always_print_percentage
def update(self, inc=1):
def update(self, inc=1, msg=''):
self._done += inc
if _NOT_TTY or IsTrace():
@ -62,12 +62,13 @@ class Progress(object):
if self._lastp != p or self._always_print_percentage:
self._lastp = p
sys.stderr.write('%s\r%s: %3d%% (%d%s/%d%s)%s' % (
sys.stderr.write('%s\r%s: %3d%% (%d%s/%d%s)%s%s%s' % (
CSI_ERASE_LINE,
self._title,
p,
self._done, self._units,
self._total, self._units,
' ' if msg else '', msg,
"\n" if self._print_newline else ""))
sys.stderr.flush()

View File

@ -58,11 +58,8 @@ else:
def _lwrite(path, content):
lock = '%s.lock' % path
fd = open(lock, 'w')
try:
with open(lock, 'w') as fd:
fd.write(content)
finally:
fd.close()
try:
platform_utils.rename(lock, path)
@ -137,6 +134,7 @@ class DownloadedChange(object):
class ReviewableBranch(object):
_commit_cache = None
_base_exists = None
def __init__(self, project, branch, base):
self.project = project
@ -150,14 +148,19 @@ class ReviewableBranch(object):
@property
def commits(self):
if self._commit_cache is None:
self._commit_cache = self.project.bare_git.rev_list('--abbrev=8',
'--abbrev-commit',
'--pretty=oneline',
'--reverse',
'--date-order',
not_rev(self.base),
R_HEADS + self.name,
'--')
args = ('--abbrev=8', '--abbrev-commit', '--pretty=oneline', '--reverse',
'--date-order', not_rev(self.base), R_HEADS + self.name, '--')
try:
self._commit_cache = self.project.bare_git.rev_list(*args)
except GitError:
# We weren't able to probe the commits for this branch. Was it tracking
# a branch that no longer exists? If so, return no commits. Otherwise,
# rethrow the error as we don't know what's going on.
if self.base_exists:
raise
self._commit_cache = []
return self._commit_cache
@property
@ -176,6 +179,23 @@ class ReviewableBranch(object):
R_HEADS + self.name,
'--')
@property
def base_exists(self):
"""Whether the branch we're tracking exists.
Normally it should, but sometimes branches we track can get deleted.
"""
if self._base_exists is None:
try:
self.project.bare_git.rev_parse('--verify', not_rev(self.base))
# If we're still here, the base branch exists.
self._base_exists = True
except GitError:
# If we failed to verify, the base branch doesn't exist.
self._base_exists = False
return self._base_exists
def UploadForReview(self, people,
auto_topic=False,
draft=False,
@ -1393,12 +1413,9 @@ class Project(object):
if is_new:
alt = os.path.join(self.gitdir, 'objects/info/alternates')
try:
fd = open(alt)
try:
with open(alt) as fd:
# This works for both absolute and relative alternate directories.
alt_dir = os.path.join(self.objdir, 'objects', fd.readline().rstrip())
finally:
fd.close()
except IOError:
alt_dir = None
else:
@ -1505,6 +1522,13 @@ class Project(object):
"""Perform only the local IO portion of the sync process.
Network access is not required.
"""
if not os.path.exists(self.gitdir):
syncbuf.fail(self,
'Cannot checkout %s due to missing network sync; Run '
'`repo sync -n %s` first.' %
(self.name, self.name))
return
self._InitWorkTree(force_sync=force_sync, submodules=submodules)
all_refs = self.bare_ref.all
self.CleanPublishedCache(all_refs)
@ -1585,7 +1609,16 @@ class Project(object):
return
upstream_gain = self._revlist(not_rev(HEAD), revid)
pub = self.WasPublished(branch.name, all_refs)
# See if we can perform a fast forward merge. This can happen if our
# branch isn't in the exact same state as we last published.
try:
self.work_git.merge_base('--is-ancestor', HEAD, revid)
# Skip the published logic.
pub = False
except GitError:
pub = self.WasPublished(branch.name, all_refs)
if pub:
not_merged = self._revlist(not_rev(revid), pub)
if not_merged:
@ -2706,41 +2739,45 @@ class Project(object):
raise
def _InitWorkTree(self, force_sync=False, submodules=False):
dotgit = os.path.join(self.worktree, '.git')
init_dotgit = not os.path.exists(dotgit)
realdotgit = os.path.join(self.worktree, '.git')
tmpdotgit = realdotgit + '.tmp'
init_dotgit = not os.path.exists(realdotgit)
if init_dotgit:
dotgit = tmpdotgit
platform_utils.rmtree(tmpdotgit, ignore_errors=True)
os.makedirs(tmpdotgit)
self._ReferenceGitDir(self.gitdir, tmpdotgit, share_refs=True,
copy_all=False)
else:
dotgit = realdotgit
try:
if init_dotgit:
os.makedirs(dotgit)
self._ReferenceGitDir(self.gitdir, dotgit, share_refs=True,
copy_all=False)
self._CheckDirReference(self.gitdir, dotgit, share_refs=True)
except GitError as e:
if force_sync and not init_dotgit:
try:
platform_utils.rmtree(dotgit)
return self._InitWorkTree(force_sync=False, submodules=submodules)
except:
raise e
raise e
try:
self._CheckDirReference(self.gitdir, dotgit, share_refs=True)
except GitError as e:
if force_sync:
try:
platform_utils.rmtree(dotgit)
return self._InitWorkTree(force_sync=False, submodules=submodules)
except:
raise e
raise e
if init_dotgit:
_lwrite(os.path.join(tmpdotgit, HEAD), '%s\n' % self.GetRevisionId())
if init_dotgit:
_lwrite(os.path.join(dotgit, HEAD), '%s\n' % self.GetRevisionId())
# Now that the .git dir is fully set up, move it to its final home.
platform_utils.rename(tmpdotgit, realdotgit)
cmd = ['read-tree', '--reset', '-u']
cmd.append('-v')
cmd.append(HEAD)
if GitCommand(self, cmd).Wait() != 0:
raise GitError("cannot initialize work tree for " + self.name)
# Finish checking out the worktree.
cmd = ['read-tree', '--reset', '-u']
cmd.append('-v')
cmd.append(HEAD)
if GitCommand(self, cmd).Wait() != 0:
raise GitError('Cannot initialize work tree for ' + self.name)
if submodules:
self._SyncSubmodules(quiet=True)
self._CopyAndLinkFiles()
except Exception:
if init_dotgit:
platform_utils.rmtree(dotgit)
raise
if submodules:
self._SyncSubmodules(quiet=True)
self._CopyAndLinkFiles()
def _get_symlink_error_message(self):
if platform_utils.isWindows():
@ -2889,13 +2926,10 @@ class Project(object):
else:
path = os.path.join(self._project.worktree, '.git', HEAD)
try:
fd = open(path)
with open(path) as fd:
line = fd.readline()
except IOError as e:
raise NoManifestException(path, str(e))
try:
line = fd.readline()
finally:
fd.close()
try:
line = line.decode()
except AttributeError:

5
repo
View File

@ -513,9 +513,8 @@ def SetupGnuPG(quiet):
sys.exit(1)
print()
fd = open(os.path.join(home_dot_repo, 'keyring-version'), 'w')
fd.write('.'.join(map(str, KEYRING_VERSION)) + '\n')
fd.close()
with open(os.path.join(home_dot_repo, 'keyring-version'), 'w') as fd:
fd.write('.'.join(map(str, KEYRING_VERSION)) + '\n')
return True

View File

@ -139,6 +139,9 @@ without iterating through the remaining projects.
p.add_option('-e', '--abort-on-errors',
dest='abort_on_errors', action='store_true',
help='Abort if a command exits unsuccessfully')
p.add_option('--ignore-missing', action='store_true',
help='Silently skip & do not exit non-zero due missing '
'checkouts')
g = p.add_option_group('Output')
g.add_option('-p',
@ -323,6 +326,10 @@ def DoWork(project, mirror, opt, cmd, shell, cnt, config):
cwd = project['worktree']
if not os.path.exists(cwd):
# Allow the user to silently ignore missing checkouts so they can run on
# partial checkouts (good for infra recovery tools).
if opt.ignore_missing:
return 0
if ((opt.project_header and opt.verbose)
or not opt.project_header):
print('skipping %s/' % project['relpath'], file=sys.stderr)

View File

@ -40,10 +40,9 @@ in a Git repository for use during future 'repo init' invocations.
helptext = self._helpDescription + '\n'
r = os.path.dirname(__file__)
r = os.path.dirname(r)
fd = open(os.path.join(r, 'docs', 'manifest-format.md'))
for line in fd:
helptext += line
fd.close()
with open(os.path.join(r, 'docs', 'manifest-format.md')) as fd:
for line in fd:
helptext += line
return helptext
def _Options(self, p):

View File

@ -51,11 +51,16 @@ class Prune(PagedCommand):
out.project('project %s/' % project.relpath)
out.nl()
commits = branch.commits
date = branch.date
print('%s %-33s (%2d commit%s, %s)' % (
print('%s %-33s ' % (
branch.name == project.CurrentBranch and '*' or ' ',
branch.name,
branch.name), end='')
if not branch.base_exists:
print('(ignoring: tracking branch is gone: %s)' % (branch.base,))
else:
commits = branch.commits
date = branch.date
print('(%2d commit%s, %s)' % (
len(commits),
len(commits) != 1 and 's' or ' ',
date))

View File

@ -315,9 +315,6 @@ later is required to fix a server side protocol bug.
# We'll set to true once we've locked the lock.
did_lock = False
if not opt.quiet:
print('Fetching project %s' % project.name)
# Encapsulate everything in a try/except/finally so that:
# - We always set err_event in the case of an exception.
# - We always make sure we unlock the lock if we locked it.
@ -350,7 +347,7 @@ later is required to fix a server side protocol bug.
raise _FetchError()
fetched.add(project.gitdir)
pm.update()
pm.update(msg=project.name)
except _FetchError:
pass
except Exception as e:
@ -371,7 +368,6 @@ later is required to fix a server side protocol bug.
fetched = set()
lock = _threading.Lock()
pm = Progress('Fetching projects', len(projects),
print_newline=not(opt.quiet),
always_print_percentage=opt.quiet)
objdir_project_map = dict()
@ -440,7 +436,7 @@ later is required to fix a server side protocol bug.
finally:
sem.release()
def _CheckoutOne(self, opt, project, lock, pm, err_event):
def _CheckoutOne(self, opt, project, lock, pm, err_event, err_results):
"""Checkout work tree for one project
Args:
@ -452,6 +448,8 @@ later is required to fix a server side protocol bug.
lock held).
err_event: We'll set this event in the case of an error (after printing
out info about the error).
err_results: A list of strings, paths to git repos where checkout
failed.
Returns:
Whether the fetch was successful.
@ -459,9 +457,6 @@ later is required to fix a server side protocol bug.
# We'll set to true once we've locked the lock.
did_lock = False
if not opt.quiet:
print('Checking out project %s' % project.name)
# Encapsulate everything in a try/except/finally so that:
# - We always set err_event in the case of an exception.
# - We always make sure we unlock the lock if we locked it.
@ -472,11 +467,11 @@ later is required to fix a server side protocol bug.
try:
try:
project.Sync_LocalHalf(syncbuf, force_sync=opt.force_sync)
success = syncbuf.Finish()
# Lock around all the rest of the code, since printing, updating a set
# and Progress.update() are not thread safe.
lock.acquire()
success = syncbuf.Finish()
did_lock = True
if not success:
@ -485,7 +480,7 @@ later is required to fix a server side protocol bug.
file=sys.stderr)
raise _CheckoutError()
pm.update()
pm.update(msg=project.name)
except _CheckoutError:
pass
except Exception as e:
@ -496,6 +491,8 @@ later is required to fix a server side protocol bug.
raise
finally:
if did_lock:
if not success:
err_results.append(project.relpath)
lock.release()
finish = time.time()
self.event_log.AddSync(project, event_log.TASK_SYNC_LOCAL,
@ -523,11 +520,12 @@ later is required to fix a server side protocol bug.
syncjobs = 1
lock = _threading.Lock()
pm = Progress('Syncing work tree', len(all_projects))
pm = Progress('Checking out projects', len(all_projects))
threads = set()
sem = _threading.Semaphore(syncjobs)
err_event = _threading.Event()
err_results = []
for project in all_projects:
# Check for any errors before running any more tasks.
@ -542,7 +540,8 @@ later is required to fix a server side protocol bug.
project=project,
lock=lock,
pm=pm,
err_event=err_event)
err_event=err_event,
err_results=err_results)
if syncjobs > 1:
t = _threading.Thread(target=self._CheckoutWorker,
kwargs=kwargs)
@ -560,6 +559,9 @@ later is required to fix a server side protocol bug.
# If we saw an error, exit with code 1 so that other scripts can check.
if err_event.isSet():
print('\nerror: Exited sync due to checkout errors', file=sys.stderr)
if err_results:
print('Failing repos:\n%s' % '\n'.join(err_results),
file=sys.stderr)
sys.exit(1)
def _GCProjects(self, projects):
@ -692,11 +694,8 @@ later is required to fix a server side protocol bug.
old_project_paths = []
if os.path.exists(file_path):
fd = open(file_path, 'r')
try:
with open(file_path, 'r') as fd:
old_project_paths = fd.read().split('\n')
finally:
fd.close()
# In reversed order, so subfolders are deleted before parent folder.
for path in sorted(old_project_paths, reverse=True):
if not path:
@ -731,12 +730,9 @@ later is required to fix a server side protocol bug.
return 1
new_project_paths.sort()
fd = open(file_path, 'w')
try:
with open(file_path, 'w') as fd:
fd.write('\n'.join(new_project_paths))
fd.write('\n')
finally:
fd.close()
return 0
def _SmartSyncSetup(self, opt, smart_sync_manifest_path):
@ -809,11 +805,8 @@ later is required to fix a server side protocol bug.
if success:
manifest_name = os.path.basename(smart_sync_manifest_path)
try:
f = open(smart_sync_manifest_path, 'w')
try:
with open(smart_sync_manifest_path, 'w') as f:
f.write(manifest_str)
finally:
f.close()
except IOError as e:
print('error: cannot write manifest to %s:\n%s'
% (smart_sync_manifest_path, e),
@ -1102,11 +1095,8 @@ class _FetchTimes(object):
def _Load(self):
if self._times is None:
try:
f = open(self._path)
try:
with open(self._path) as f:
self._times = json.load(f)
finally:
f.close()
except (IOError, ValueError):
try:
platform_utils.remove(self._path)
@ -1126,11 +1116,8 @@ class _FetchTimes(object):
del self._times[name]
try:
f = open(self._path, 'w')
try:
with open(self._path, 'w') as f:
json.dump(self._times, f, indent=2)
finally:
f.close()
except (IOError, TypeError):
try:
platform_utils.remove(self._path)

View File

@ -271,11 +271,6 @@ Gerrit Code Review: https://www.gerritcodereview.com/
branches[project.name] = b
script.append('')
script = [ x.encode('utf-8')
if issubclass(type(x), unicode)
else x
for x in script ]
script = Editor.EditString("\n".join(script)).split("\n")
project_re = re.compile(r'^#?\s*project\s*([^\s]+)/:$')

60
tests/test_editor.py Normal file
View File

@ -0,0 +1,60 @@
# -*- coding:utf-8 -*-
#
# Copyright (C) 2019 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Unittests for the editor.py module."""
from __future__ import print_function
import unittest
from editor import Editor
class EditorTestCase(unittest.TestCase):
"""Take care of resetting Editor state across tests."""
def setUp(self):
self.setEditor(None)
def tearDown(self):
self.setEditor(None)
@staticmethod
def setEditor(editor):
Editor._editor = editor
class GetEditor(EditorTestCase):
"""Check GetEditor behavior."""
def test_basic(self):
"""Basic checking of _GetEditor."""
self.setEditor(':')
self.assertEqual(':', Editor._GetEditor())
class EditString(EditorTestCase):
"""Check EditString behavior."""
def test_no_editor(self):
"""Check behavior when no editor is available."""
self.setEditor(':')
self.assertEqual('foo', Editor.EditString('foo'))
def test_cat_editor(self):
"""Check behavior when editor is `cat`."""
self.setEditor('cat')
self.assertEqual('foo', Editor.EditString('foo'))

View File

@ -18,11 +18,30 @@
from __future__ import print_function
import contextlib
import os
import shutil
import subprocess
import tempfile
import unittest
import git_config
import project
@contextlib.contextmanager
def TempGitTree():
"""Create a new empty git checkout for testing."""
# TODO(vapier): Convert this to tempfile.TemporaryDirectory once we drop
# Python 2 support entirely.
try:
tempdir = tempfile.mkdtemp(prefix='repo-tests')
subprocess.check_call(['git', 'init'], cwd=tempdir)
yield tempdir
finally:
shutil.rmtree(tempdir)
class RepoHookShebang(unittest.TestCase):
"""Check shebang parsing in RepoHook."""
@ -60,3 +79,58 @@ class RepoHookShebang(unittest.TestCase):
for shebang, interp in DATA:
self.assertEqual(project.RepoHook._ExtractInterpFromShebang(shebang),
interp)
class FakeProject(object):
"""A fake for Project for basic functionality."""
def __init__(self, worktree):
self.worktree = worktree
self.gitdir = os.path.join(worktree, '.git')
self.name = 'fakeproject'
self.work_git = project.Project._GitGetByExec(
self, bare=False, gitdir=self.gitdir)
self.bare_git = project.Project._GitGetByExec(
self, bare=True, gitdir=self.gitdir)
self.config = git_config.GitConfig.ForRepository(gitdir=self.gitdir)
class ReviewableBranchTests(unittest.TestCase):
"""Check ReviewableBranch behavior."""
def test_smoke(self):
"""A quick run through everything."""
with TempGitTree() as tempdir:
fakeproj = FakeProject(tempdir)
# Generate some commits.
with open(os.path.join(tempdir, 'readme'), 'w') as fp:
fp.write('txt')
fakeproj.work_git.add('readme')
fakeproj.work_git.commit('-mAdd file')
fakeproj.work_git.checkout('-b', 'work')
fakeproj.work_git.rm('-f', 'readme')
fakeproj.work_git.commit('-mDel file')
# Start off with the normal details.
rb = project.ReviewableBranch(
fakeproj, fakeproj.config.GetBranch('work'), 'master')
self.assertEqual('work', rb.name)
self.assertEqual(1, len(rb.commits))
self.assertIn('Del file', rb.commits[0])
d = rb.unabbrev_commits
self.assertEqual(1, len(d))
short, long = next(iter(d.items()))
self.assertTrue(long.startswith(short))
self.assertTrue(rb.base_exists)
# Hard to assert anything useful about this.
self.assertTrue(rb.date)
# Now delete the tracking branch!
fakeproj.work_git.branch('-D', 'master')
rb = project.ReviewableBranch(
fakeproj, fakeproj.config.GetBranch('work'), 'master')
self.assertEqual(0, len(rb.commits))
self.assertFalse(rb.base_exists)
# Hard to assert anything useful about this.
self.assertTrue(rb.date)