All the stdlib imports are supposed to come before any local imports.
Change-Id: I10c0335ba2ff715fd34c9eb91bfe6560e904df08
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255593
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
This allows us to control sync output better by having three levels
of output: quiet (only errors), default (progress bars), verbose (all
the things). For now, we just put the chatty "already have persistent
ref" message behind the verbose level.
Bug: https://crbug.com/gerrit/11293
Change-Id: Ia61333fd8085719f3e99edb7b466cdb04031b67f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255414
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
All of the instances of this are related to Python 2 names that
don't exist in Python 3, and the warnings are raised when running
flake8 on Python 3.
All of these will go away once we completely remove support for
Python 2, so just suppress them inline. We don't globally suppress
the check so that we will still see legitimate errors if/when they
occur in new code.
Change-Id: Iccf955f50abfc9f83b371fc0af6cceb51037456f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255039
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
We increased the max line length to 100 columns which got rid of
the majority of these warnings, but there were still a few lines
that exceeded 100 columns.
Change-Id: Ib3372868ca2297f83073a14f91c8ae3df9d0d0e6
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254699
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
Some people find the existing output to be a bit confusing. It spews
a lot of git output before exiting, but it's not exactly clear what
the final state is when things pass. Add an explicit message.
Bug: https://crbug.com/gerrit/10501
Change-Id: I9de83b595d3185feb820005b8fc81c6adc55b357
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254732
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
We've been requiring git-1.7.2 since Oct 2012, so we can safely drop
the individual checks sprinkled throughout the code base for older.
Change-Id: I1737fff7b3f27f475960b0bff9cb300aefd5d108
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253135
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
flake8 reports:
E722 do not use bare 'except'
Replace them with 'except Exception' per [1] which says:
Bare except will catch exceptions you almost certainly don't want
to catch, including KeyboardInterrupt (the user hitting Ctrl+C) and
Python-raised errors like SystemExit
If you don't have a specific exception you're expecting, at least
except Exception, which is the base type for all "Regular" exceptions.
[1] https://stackoverflow.com/a/54948581
Change-Id: Ic555ea9482645899f5b04040ddb6b24eadbf9062
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254606
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
- E301 expected 1 blank line
- E302 expected 2 blank lines
- E303 too many blank lines
- E305 expected 2 blank lines after class or function definition
- E306 expected 1 blank line before a nested definition
Fixed automatically with autopep8:
git ls-files | grep py$ | xargs autopep8 --in-place \
--select E301,E302,E303,E305,E306
Manually fix issues in project.py caused by misuse of block comments.
Change-Id: Iee840fcaff48aae504ddac9c3e76d2acd484f6a9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254599
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
- E121 continuation line under-indented for hanging indent
- E122 continuation line missing indentation or outdented
- E125 continuation line with same indent as next logical line
- E126 continuation line over-indented for hanging indent
- E127 continuation line over-indented for visual indent
- E128 continuation line under-indented for visual indent
- E129 visually indented line with same indent as next logical line
- E131 continuation line unaligned for hanging indent
Fixed automatically with autopep8:
git ls-files | grep py$ | xargs autopep8 --in-place \
--select E121,E122,E125,E126,E127,E128,E129,E131
Change-Id: Ifd95fb8e6a1a4d6e9de187b5787d64a6326dd249
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254605
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: David Pursehouse <dpursehouse@collab.net>
flake8 reports:
E713 test for membership should be 'not in'
Change-Id: I4446be67c431b7267105b53478d2ceba2af758d7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254451
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
flake8 reports:
E714 test for object identity should be 'is not'
Change-Id: Ib8c4100babaf952bbfe65fd56555ece8a958e4b0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254450
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
flake8 reports:
E265 block comment should start with '# '
While we're at it, add a period at the end of the comment sentence.
Change-Id: Icb7119079a1d64e6defafc3f6d24e99dbf16139d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254596
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
When upload hooks fail, people are forced to use --no-verify to upload
CLs anyways. When projects have flaky hooks, this trains people to
always use that option. This is obviously bad: hooks might get fixed,
or some of the hooks are always good & people should review.
Lets add an --ignore-hooks option. This still runs the hooks, but any
failures will be ignored and allow the user to upload anyways.
Bug: https://crbug.com/gerrit/12230
Change-Id: Ide2ac8a40a656bfcd6aae20c3ce8118e06bf909b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254452
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
flake8 reports:
W391 blank line at end of file
Change-Id: I5498b2de2d1268d4f1f4b9e1760f9fa93a6da4cd
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254594
Tested-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
We were perhaps a bit too hasty to jump to git-2.10. Existing LTS
releases of Ubuntu are quite old still: Trusty has 1.9 while Xenial
has 2.5. While we plan on dropping support for those eventually as
we migrate to Python 3.6, we don't need to be so strict just yet on
the git versions.
We also want to disconnect the version the repo launcher requires
from the version the rest of the source tree requires. The repo
launcher doesn't need as many features, and being flexible there
allows us more freedom to upgrade & rollback as needed.
So we'll allow git-1.7 again, but start warning on any users older
than git-1.9. This aligns better with existing LTS releases, and
gives users a chance to start upgrading before we cut them off.
Change-Id: I140305dd8e42c9719c84e2aee0dc6a5c5b18da25
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254573
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Some automatic git operations will prune objects on us, and not just
the gc step. Normally we don't care, but with shared projects, we
will have multiple git checkouts with refs that the others cannot
see, but with a shared object dir. Any pruning of objects based on
refs in just one repo can easily break the others.
git-2.7.0 introduced a preciousObjects setting which tells git to
never prune objects for this exact scenario: there might be refs in
some location that git is unable to see.
Change-Id: I781de27c5bbe1d4c70f0187566141c9cce088bd8
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254392
Reviewed-by: Nasser Grainawi <nasser@codeaurora.org>
Reviewed-by: David Riley <davidriley@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
Execution of 'repo forall -p -c' doesn't work with Py3 and ends up
with an error:
Got an error, terminating the pool: TypeError: can only concatenate
str (not "bytes") to str
That's fixed by using the decode() method.
Change-Id: Ice01aaa1822dde8d957b5bf096021dd5a2b7dd51
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253659
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Jiri Tyr <jiri.tyr@gmail.com>
These files are not directly executable, so drop the +x bits.
Change-Id: Iaf19a03a497686cc21103e7ddf08073173440dd1
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254076
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Instead of not using '-c' for '--current-branch' when using gitc, we
were only using '-c' when using gitc, so we still had the conflict with
the gitc option, and other users still couldn't use '-c'.
Test: repo init -u https://android.googlesource.com/platform/manifest; repo init -c
Test: repo gitc-init -u ... -b ... -c testing
Change-Id: I71e4950a49c281418249f0783c6a2ea34f0d3e2b
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253795
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Dan Willemsen <dwillemsen@google.com>
Currently our default behavior is:
* Try to sync all repos
* If any errors seen, exit
* Try to garbage collect all repos
* If any errors seen, exit
* Try to update local project list
* If any errors seen, exit
* Try to checkout out all local repos
* If any errors seen, exit
Users find these incomplete syncs confusing, so lets try to complete
as much as possible by default and printing out summaries at the end.
Bug: https://crbug.com/gerrit/11293
Change-Id: Idd17cc9c3bbc574d8a0f08a30225dec7bfe414cb
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/238554
Reviewed-by: Michael Mortensen <mmortensen@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The use case is any situation where your manifest does
not exist on server, but where you still want to do
full sync for the projects, without having your
workspace manifest switched to other branch or
forwarded to latest or similar.
This allows syncing to a historical manifest in git log,
that does not have a branch, as well as when integrating
something together that has not been pushed upstream yet.
Changes can also exist locally on a manifest that is
behind head, meaning not requiring rebase to latest.
Tested using:
$ cd .repo/manifests/
$ git checkout <any hash 1>
$ <do local modifications>
$ repo sync --no-manifest-update
$ git checkout <any hash 2>
$ repo sync --no-manifest-update
Change-Id: I0c9773aa8bc5876813a2e7d7fec697abcb2d9e94
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/246445
Tested-by: Fredrik de Groot <fredrik.de.groot@volvocars.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
We keep getting requests for init to support -c. This conflicts with
gitc-init which allocates -c for its own use. Lets make this dynamic
so we keep it with "init" but omit it for "gitc-init".
Bug: https://crbug.com/gerrit/10200
Change-Id: Ibf69c2bbeff638e28e63cb08926fea0c622258db
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253252
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This makes it consistent with the short option for current-branch in
repo sync.
Change-Id: I2848e87f45a66ef8d829576d0c0c4c0f7a8636a0
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/241700
Tested-by: Diogo Ferreira <deovferreira@gmail.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
After Ib546f5ebbc8a23875fbd14bf166fbe95b7dd244e, repo info now displays
the current project revision in the 'Current revision' field.
While the output is more consistent, there are use cases for the
revision expression as shown in the manifest. This patch re-adds the
manifest revision as a new 'Manifest revision' field.
Change-Id: I50c1559dcb7ceb69af07352b956d78f85b8f592e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/240799
Tested-by: Diogo Ferreira <deovferreira@gmail.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
No functional changes, just unifying duplicate code paths.
Change-Id: I6afa797ca1e1eb90abdc0236325003ae070cbfb3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/247293
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
The upload module tries to turn the strings into bytes before passing
to EditString, but it combines bytes & strings causing an error. The
return value might be bytes or string, but the caller only expects a
string. Lets simplify this by sticking to strings everywhere and have
EditString take care of converting to/from bytes when reading/writing
the underlying files. This also avoids possible locale confusion when
reading the file by forcing UTF-8 everywhere.
Bug: https://crbug.com/gerrit/11929
Change-Id: I07b146170c5e8b5b0500a2c79e4213cd12140a96
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/245621
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Series of steps:
* Create a local "b1" branch with `repo start b1` that tracks a remote
branch (totally fine)
* Manually create a local "b2" branch with `git branch --track b1 b2`
that tracks the local "b1" (uh-oh...)
* Delete the local "b1" branch manually or via `repo prune` (....)
* Try to process the "b2" branch with `repo prune`
Since b2 tracks a branch that no longer exists, everything blows up
at this point as we try to probe the non-existent ref. Instead, we
should flag this as unknown and leave it up to the user to resolve.
This probably could come up if a local branch was tracking a remote
branch that was deleted from the server, and users ran something like
`repo sync --prune` which cleaned up the remote refs.
Bug: https://crbug.com/gerrit/11485
Change-Id: I6b6b6041943944b8efa6e2ad0b8b10f13a75a5c2
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/236793
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Kirtika Ruchandani <kirtika@google.com>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
The current sync output displays "Fetching project" and "Checking out
project" messages and progress bar updates independently leading to a
lot of spam. Lets merge these periodic outputs with the status bar to
get a little bit tighter output in the normal case. This doesn't solve
all our problems, but gets us closer.
Bug: https://crbug.com/gerrit/11293
Change-Id: Icd627830af4dd934a9355b7ace754b56dc96cfef
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244934
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
When repo sync fails because some git trees are not in clean state and
as such can not be rebased automatically, it is a pain to figure out
which trees are the culprits.
With this patch the list of offending trees is printed when repo sync
reports checkout errors.
TEST=ran 'repo sync' and observed the proper list of directories show
up after the final error message
Bug: https://crbug.com/gerrit/11293
Change-Id: Icdf1a03e9014ecb184f331f513cc9a2efc7d11ed
Signed-off-by: Vadim Bendebury <vbendeb@google.com>
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244053
Reviewed-by: Mike Frysinger <vapier@google.com>
Use open() as a context manager to simplify the close logic and make
the code easier to read & understand. This is also more Pythonic.
Change-Id: I579d03cca86f99b2c6c6a1f557f6e5704e2515a7
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244734
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
In CrOS, our infra has to deal with partial checkouts constantly
(for a variety of reasons). To help reset back to a good state,
we run git commands via `repo forall`, but don't care about the
missing checkouts. Add a flag so we can disambiguate between
missing repos and failing git subcommands.
Bug: https://crbug.com/1013377
Bug: https://crbug.com/1013623
Change-Id: Ie3498c6d111276c60d2ecedbba21bfa778588d50
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/241935
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
The "Current revision" field shows the revision as listed in the
manifest. I think most users expect this to show the revision
that the git tree is checked out to instead. Switch the output
to show that revision instead, and add a "Current branch" if it
matches a local branch.
Change-Id: Ib546f5ebbc8a23875fbd14bf166fbe95b7dd244e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239240
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
If the user passes in bad projects like `repo info asdf`, we currently
silently swallow those and do nothing. Allow NoSuchProjectError to
bubble up to main which will handle & triage this correctly for us.
Change-Id: Ie04528e7b7a164293063a636813a73eaabdd5bc3
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239238
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Depending on where/how repo is invoked, the active version might be
from a git tree, and it might be different from the .repo/repo.git/
version in the current repo client checkout. Report both if they're
different so it's clearer.
Lets also include the two different User-Agent's that we set up when
talking to networked services.
Bug: https://crbug.com/gerrit/11144
Change-Id: I2ebb6e3ac30e374a8406cab3e4438087246a8c57
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239234
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
If `git grep` fails in any project checkout (e.g. an incomplete
sync), make sure we print that error clearly rather than blowing
up, and exit non-zero in the process.
Bug: https://crbug.com/gerrit/11613
Change-Id: I31de1134fdcc7aaa9814cf2eb6a67d398eebf9cf
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239237
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
If `git diff` fails in any project checkout (e.g. an incomplete
sync), make sure we print that error clearly rather than blowing
up, and exit non-zero in the process.
Bug: https://crbug.com/gerrit/11613
Change-Id: I12f278427cced20f23f8047e7e3dba8f442ee25e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239236
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
If a repo doesn't exist (e.g. an incomplete sync), make sure we exit
non-zero when they get skipped.
Change-Id: Ifff711e374416b1e6b9b8da4fdc6f14b27ced450
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/239235
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This uses coloring style like we use in grep/forall already.
Change-Id: I317e2e47567a30c513083c48e7c7c40b091bb29a
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/238555
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Lets switch the default rebase behavior to align with our new sync
behavior: we try to rebase all projects by default and exit/summarize
things at the very end if there were any errors. Or if people want
to exit immediately, they can use the new --fail-fast option.
Change-Id: I436ac563f972b45de6ce9ad74da1e4870e584902
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/238553
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Cut out some more standalone code from Execute to make this func a
bit more manageable. The manifest project update is pretty simple
and standalone, but still takes up a good chunk of what's left.
Change-Id: Idc2442d9def495eccd0a49cda203c44aef16f129
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/236614
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
The smart sync logic takes up about 45% of the overall Execute func
and is about 100 lines of code. The only effect it has on the rest
of the code is to set the manifest_name variable. Since this func
is already quite huge, split the smart sync logic out.
Change-Id: Id861849b0011ab47387d74e92c2ac15afcc938ba
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234835
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
This makes it way easier to recover from forgetting to run repo start
before committing: just run `repo start -b new-branch`, instead of
all that tedious mucking around with reflogs.
Change-Id: I56d49dce5d027e28fbba0507ac10cd763ccfc36d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/232712
Reviewed-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
In commit d9e5cf0e ("sync: invert --force-broken with --fail-fast") the
force-broken option has been deprecated. Accidentally the option has
been changed from Boolean to Value. This breaks all users of repo with:
main.py: error: -f option requires an argument
This is easy to avoid by keeping the type.
Signed-off-by: Stefan Müller-Klieser <s.mueller-klieser@phytec.de>
Change-Id: Ia8b589cf41ac756d10c61e17ec8d76ba8f7031f9
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/235043
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Mike Frysinger <vapier@google.com>
A common pattern in our subcommands is to verify the arguments &
options before executing things. For some subcommands, that check
stage is quite long which makes the execution function even bigger.
Lets split that logic out of the execute phase so it's easier to
manage these.
This is most noticeable in the sync subcommand whose Execute func
is quite large, and the option checking makes up ~15% of it.
The manifest command's Execute can be simplified significantly as
the optparse configuration always sets output_file to a string.
Change-Id: I7097847ff040e831345e63de6b467ee17609990e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234834
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This is useful when you want to scan all the possibilities of repo
at once. Like when you're searching for different option names.
Change-Id: I225dfb94d2be78229905b744ecf57eb2829bb52d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/232894
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
People seem to not expect the sync process to halt immediately if an
error is encountered. It's also basically guaranteed to leave their
tree in an incomplete state. Lets invert the default behavior so we
attempt to sync (both fetch & checkout) all projects. If an error is
hit, we still exit(1) and show it at the end.
If people want the sync to abort quickly, they can use the new option
--fail-fast.
Bug: https://crbug.com/gerrit/11293
Change-Id: I49dd6c4dc8fd5cce8aa905ee169ff3cbe230eb3d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/234812
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
This makes the code a bit easier to read by doing all the project
independent settings first instead of repeating it for every for
loop iteration.
Change-Id: I4ff21296e444627beba2f4b86561069f5e9a0d73
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233554
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
Callers don't actually see -1 (they'll usually see 255, but the exact
answer here is complicated). Just switch to 1 as that's the standard
value tools use to indicate an error.
Change-Id: Ib712db1924bc3e5f7920bafd7bb5fb61f3bda44f
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233553
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
The partial clone rework (commit 745be2ede1
"Add support for partial clone") changed the behavior when a single repo
hit a failure: it would always call sys.exit() immediately. This isn't
even necessary as we already pass down an error event object which the
workers set and the parent checks. Just delete the exit entirely.
Change-Id: Id72d8642aefa2bde24e1a438dbe102c3e3cabf48
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/233552
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
A new option, --partial-clone is added to 'repo init' which tells repo
to utilize git's partial clone functionality, which reduces disk and
bandwidth usage when downloading by omitting blob downloads initially.
Different from restricting clone-depth, the user will have full access
to change history, etc., as the objects are downloaded on demand.
Change-Id: I60326744875eac16521a007bd7d5481112a98749
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/229532
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: Xin Li <delphij@google.com>
We're relying on sys.stdout.write() to flush its buffer which isn't
guaranteed, and is not the case in Python 3. Change to use print()
everywhere to be standard, and utilize the end= keyword to get the
EOL semantics we need.
We can't use print's flush= keyword as that's only in Python 3.
Leave behind a TODO to clean it up when we can drop Python 2.
Bug: https://crbug.com/gerrit/10418
Change-Id: I562128c7f1e6d154f4a6ecdf33a70fa2811dc2af
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/230392
Tested-by: Mike Frysinger <vapier@google.com>
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Since ParseGitVersion can call `git --version` automatically, we don't
need this duplicate version() helper anymore. The only other user is
the `repo version` code, so convert that to version_tuple().full.
Bug: https://crbug.com/gerrit/11144
Change-Id: I9d77822fc39f4ba28884d9183359169cabf5f17d
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/231055
Reviewed-by: David Pursehouse <dpursehouse@collab.net>
Tested-by: Mike Frysinger <vapier@google.com>
%prog represents the full subcommand ("repo" + subcommand name), not a
Windows-style environment variable for "repo". The current help output
shows
repo forall% forall ...
Correct the variable usage so it shows "repo forall ..." instead.
Change-Id: I1fea55572428cc922ddf24ace1168a3d8f82dad0
Neither of the fields here expect floats so make sure we use integer
division when calculating things.
Bug: https://crbug.com/gerrit/10418
Change-Id: Ibda068b16a7bba7ff3efba442c4bbff4415caa6e
This module uses print() so make sure we import the print function.
It doesn't really impact the current code due to the simple way it
is calling print, but we should be sane to avoid future issues.
Change-Id: I0b15344678c1dcb71207faa333c239b3fced1d62
There's no reason to support any other encoding in these files.
This only affects the files themselves and not streams they open.
Bug: https://crbug.com/gerrit/10418
Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
Use the `raise` statement directly.
Switch to using .items() instead of .iteritems(). Python 3 doesn't
have .iteritems() as .items() is a generator, and these are small
enough that the Python 2 overhead should be negligible.
We have to run .keys() through list() in a few places as Python 3
uses a generator and we sometimes want to iterate more than once.
That's why we don't change all .keys() or .items() calls -- most
are in places where generators are fine.
Bug: https://crbug.com/gerrit/10418
Change-Id: I469899d9b77ffd77ccabb831bc4b217407fefe6f
The current implementation ignores the user-specified paths to
manifests. if the "repo diffmanifests" is invoked with absolute
file paths for one or both manifests, the command fails with message:
fatal: duplicate path ... in /tmp/manifest-old.xml
Also the current implementation fails to expand the absolute path to
manifest files if "repo diffmanifests" is invoked with relative
paths, i.e "repo diffmanifests manifest-old.xml manifest-new.xml".
fatal: manifest manifest-old.xml not found
This commit fixes the first issue by disabling the local manifest
discovery for diffmanifests command, and the second issue by
expanding paths to manifests within "diffmanifests" sub-command.
Test: repo manifest --revision-as-HEAD -o /tmp/manifest-old.xml
repo sync
repo manifest --revision-as-HEAD -o /tmp/manifest-new.xml
repo diffmanifests /tmp/manifest-old.xml /tmp/manifest-new.xml
Change-Id: Ia125d769bfbea75adb9aba81abbd8c636f2168d4
Signed-off-by: Vasyl Gello <vasek.gello@gmail.com>
If a user is asking for a shallow clone of the repos, they probably
expect a shallow clone of the manifest repo too. For very large
manifest repos, this can be a huge space and time savings. For one real-world
repo, a 'repo init --no-tags --current-branch' used 350MB of disk space and
took 7 minutes. Adding --depth 1 and this change reduced it to 10MB and 2.5
minutes.
Change-Id: I6fa662e174e623ede8861efc862ce26d65d4958d
If "repo init" was run in a path containing "%", "repo info" would fail
printing the path with
File ".repo/repo/color.py", line 173, in f
return fmt % args
TypeError: not enough arguments for format string
as the "%" in the path name is interpreted as the start of a formatting
specifier. Avoid that by using the non-formatting printer for headtext
which does not require any formatting so there is no need to try to
expand "%" sequences.
Change-Id: Ie193b912191fe7cdabdce5c97bb100f0714f6e76
Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>
Forcefully remove dirty projects if option '--force-remove-dirty' is given.
The '--force-remove-dirty' option can be used to remove previously used
projects with uncommitted changes. WARNING: This may cause data to be lost
since uncommitted changes may be removed with projects that no longer exist
in the manifest.
Change-Id: I844a6e943ded522fdc7b1b942c0a1269768054bc
When --ne/--no-emails is added to 'repo upload' command line, gerrit
server will not generate notification emails.
project.py:Project.UploadForReview method is modified to accept a
string recognizable by gerrit to indicate different sets of destination
email addressees, but the upload command line allows only one option -
disable sending emails completely.
Default repo upload behavior is not being changed.
TEST=tried in the Chrome OS repo, observed that patches uploaded with
--ne or --no-emails indeed do not trigger any emails, while
patches uploaded without these command line options still trigger
email notifications.
Change-Id: I0301edec984907aedac277d883bd0e6d3099aedc
* Add more file i/o wrappers in platform_utils to allow using
long paths (length > MAX_PATH) on Windows.
* Paths using the long path syntax ("\\?\" prefix) should never
escape the platform_utils API surface area, so that this
specific syntax is not visible to the rest of the repo code base.
* Forward many calls from os.xxx to platform_utils.xxx in various place
to ensure long paths support, specifically when repo decides to delete
obsolete directories.
* There are more places that need to be converted to support long paths,
this commit is an initial effort to unblock a few common use cases.
* Also, fix remove function to handle directory symlinks
Change-Id: If82ccc408e516e96ff7260be25f8fd2fe3f9571a
"repo init --reference" has two purposes: to decrease bandwidth used
at clone time, and to decrease disk usage afterward, by using the
reference repositories as an alternate store of objects even after
the clone. The downside is that it makes the borrowing repositories
dependent on the reference repositories, so it is easy to end up
with missing objects by mistake after a cleanup operation like "git
gc".
To prevent that, v2.3.0-rc0~30^2 (clone: --dissociate option to mark
that reference is only temporary, 2014-10-14), "git clone" gained a
--dissociate option that makes --reference reuse objects from the
reference repository at clone time but copy them over instead of
using the reference as an alternate. This is more straightforward to
use than plain --reference, at the cost of higher disk usage.
Introduce a --dissociate to "repo init" that brings the same benefits
to repo. The option is simply passed on to "git clone".
Change-Id: Ib50a549eb71e0a2b3e234aea57537923962a80d4
Since gitiles recommends using # headers over ---/=== underlines,
change the manifest-format.md over and all our help texts.
Change-Id: I96391d41fba769e9f26870d497cf7cf01c8d8ab3
pylint is not used since bb5b1a0. The pyflakes cleanup mentioned in that
commit has not been done, but given that this project is no longer being
actively developed I don't think it's worth spending time doing it.
Leaving the pylint suppressions causes confusion because it leads people
to think that we are still using pylint.
Change-Id: If7d9f280a0f408c780f15915ffdb80579ae21f69
When someone does "repo download -c <project> <change>"
without specifying a patch number, by default patch 1 is
downloaded. An alternative is to look for the latest patch
and download the same when no explicit patch is given.
This commit does the same by identifying the latest patch
using "git ls-remote".
Change-Id: Ia5fa7364415f53a3d9436df4643e38f3c90ded58
Put the correctly-expanded relative paths in objects/info/alternates.
From gitrepository-layout(5), this path should be "relative to the
object database, not to the repository".
Change-Id: I7b2027ae23cf7d367b80f5a187603c4cbacdb2de
If a user executes:
repo forall -c echo $REPO_PROJECT
then $REPO_NAME is expanded by the user's shell first, and passed
as $1 to the shell that executes echo. This will either result in
no output, or output of whatever REPO_NAME is set to in the user's
shell. Either way, this is an unexpected result.
The correct way to do it is:
repo forall -c 'echo $REPO_PROJECT'
such that $REPO_NAME is passed in to the shell literally, and then
expanded to the value set in the environment that was passed to
the shell.
Update the documentation to make this clearer.
Change-Id: I713caee914172ad8d8f0fafacd27026502436f0d
os.remove raises an exception when deleting read-only files on
Windows. Replace all calls with calls to platform_utils.remove,
which deals with deals with that issue.
Change-Id: I4dc9e0c9a36b4238880520c69f5075eca40f3e66
* changes:
Port os.rename calls to work on Windows
Workaround shutil.rmtree limitation on Windows
Add support for creating symbolic links on Windows
Make "git command" and "forall" work on Windows
When starting a branch, branch.merge is set to project revision unless
the revision is a SHA1. In that case, branch.merge is set to dest_branch
if defined or manifest default revision otherwise. This special handling
allows repo upload to work when the project revision is a SHA1.
Extend the special handling to also happen when the project revision
is a tag value or a change value so that repo upload will work in those
case as well.
Change-Id: Iff81ece40e770cd02535e80dcb023564d42dcf47
This option allow to bypass verification ssl certification while
establishing connection with Gerrit to upload review.
Change-Id: If2e15f5a273c18a700eb5093ca8a4d5a4cbf80cd
This reverts commit d88f53e2b9. I merged
it too hastily without paying enough attention to compatibility with
released Gerrit versions.
Change-Id: I4028d4737df1255f11e217da183a19a010597d5b
Considering that some users might expect changes created with
'-d' option are not public. Private changes may be a better
choice here than work-in-progress changes.
Change-Id: I46a8fb9ae38beb41cf96d6abe82bea6db2439669
This change adds options for git-repo tool to support private
changes and work-in-progress changes.
Change-Id: I343491f5949f06f1580d53f9cc0dee2dca09130f
$ git ls-files | grep py$ | xargs pyflakes
subcmds/stage.py:101: list comprehension redefines 'p' from line 63
subcmds/sync.py:784: list comprehension redefines 'p' from line 664
subcmds/upload.py:467: list comprehension redefines 'avail' from line 454
Change-Id: Ia65d1a72ed185ab3357e1a91ed4450c719e75a7c
The submodule argument to Sync_LocalHalf was missing in
MetaBranchSwitch, causing submodules not to get synced when the
-b/--manifest-branch argument to init is used.
Change-Id: Ie86d271abac2020725770be36ead83be3326e64b
Signed-off-by: Martin Kelly <mkelly@xevo.com>
With --force-broken it continue to fetch other projects but nothing
is added in directory because it abort some lines later.
Change-Id: I32c4a4619b3028893dc4f98e8d4e5bc5c09adb27
By default, shutil.rmtree raises an exception when deleting readonly
files on Windows.
Replace all shutil.rmtree with platform_utils.rmtree, which adds an
error handler to make files read-write when they can't be deleted.
Change-Id: I9cfea9a7b3703fb16a82cf69331540c2c179ed53
Python on Windows does not support non blocking file operations.
To workaround this issue, we instead use Threads and a Queue to
simulate non-blocking calls. This is happens only when running
with the native Windows version of Python, meaning Linux and Cygwin
are not affected by this change.
Change-Id: I4ce23827b096c5138f67a85c721f58a12279bb6f
As reported by pyflakes:
subcmds/abandon.py:84: undefined name 'p'
The name of the variable should be 'proj'.
Change-Id: Ic09eb92e8db6b510e99efce010bd0bb094d7cbfe
repo sync can sync submodules via the --fetch-submodules option.
However, if the manifest repo has submodules, those will not be synced.
Having submodules in the manifest repo -- while not commonly done -- can
be useful for inheriting a manifest from another project using <include>
and layering changes on top of it. In this way, you can avoid having to
deal with merge conflicts between your own manifests and the other
project's manifests (for example, if you're managing an Android fork).
Add a --submodule option to init that automatically syncs the submodules
in the manifest repo whenever the manifest repo changes.
Change-Id: I45d34f04517774c1462d7f233f482d1d81a332a8
Signed-off-by: Martin Kelly <mkelly@xevo.com>
repo can be configured to download from any number of remote git repos.
However when one fails repo doesn't report which one. Example:
Fatal: remote error: Daily ls-remote rate limit exceeded for IP xx.xx.xx.xx
TEST=repo init -q -u https://chromium.googlesource.com/chromiumos/manifest.git
# Apply patch in ./.repo/repo/
# Simulate a git remote error:
sed -i -e 's#chromiumos/docs#chromiumos/XXdocs#' .repo/manifests/full.xml
repo sync --quiet --force-sync docs
# error message now shows the remote URL
Optional test tip: reduce the time.sleep(random(...)) in ./.repo/repo/project.py
Change-Id: I4509383b6a43a8e66064778e8ed612d8a735c8b6
The --quiet option reduces the output to just
a list of projects with modified workspaces (and
orphans if -o is specified)
A common use case is when performing a full-workspace
merge. The integrator will kick-off a merge via:
repo forall -c git merge <some tag>
And then produce a short list of conflicted projects via:
repo status -q
The integrator can then iteratively fix and clean up all conficted
components. The merge is complete when:
repo status -q
returns no output.
Change-Id: Ibbba8713eac35befd8287c95948874e23fd5c7e2
when you want to delete all local branches, you should be find
all branches' name, and type them behind 'repo abandon' command.
Usage:
repo abandon --all [<project>...]
Change-Id: I4d391f37fb9d89b8095488c585468eafc1a35f31
When repo syncs a manifest that utilizes multiple branches
in the same project, then the sync will use an extra
thread for each "duplicate". For example, if
the manifest includes the project "foo" and "bar"
twice, then "repo sync -jN" will fetch with N+2 threads.
This is caused by _FetchHelper() releasing the thread semaphore
object each time it's called, even though _FetchProjectList()
may call this function multiple times within the scope of a
single thread.
Fix by moving the thread semaphore release to
_FetchProjectList(), which is only called once per thread
instance.
Change-Id: I1da78b145e09524d40457db5ca5c37d315432bd8
When there's a symlink to a directory, os.walk still lists the symlink
in dirs, even if it isn't configured to follow symlinks. This will fail
the listdirs check if the symlink is broken (either before or during the
cleanup). So instead, check for directory symlinks and remove them using
os.remove.
Bug: Issue 231
Change-Id: I0ec45a26be566613a4a39bf694a3d9c6328481c2
When there are nested projects in a manifest, like on AOSP right now:
<project path="build" name="platform/build" />
<project path="build/blueprint" name="platform/build/blueprint" />
<project path="build/kati" name="platform/build/kati" />
<project path="build/soong" name="platform/build/soong" />
And the top "build" project is removed (or renamed to remove the
nesting), repo just wipes away everything under build/ and re-creates
the projects that are still there. But it only checks to see if the
build/ project is dirty, so if there are dirty files in a nested
project, they'll just be blown away, and a fresh worktree checked out.
Instead, behave similarly to how `git clean -dxf` behaves and preserve
any subdirectories that have git repositories in them. This isn't as
strict as git -- it does not check to see if the '.git' entry is a
readable gitdir, just whether an entry named '.git' exists.
If it encounters any errors removing files, we'll print them all out to
stderr and tell the user that we were unable to clean up the obsolete
project, that they should clean it up manually, then sync again.
Change-Id: I2f6a7dd205a8e0b7590ca5369e9b0ba21d5a6f77
The shared object stores confuse git and make it throw away objects which are
still in use. We'll avoid that problem by disabling automatic pruning on those
projects, but there's nothing preventing a user from changing the config back
or pruning a repository manually.
BUG=chromium:375945
TEST=Ran repo sync on fresh ChromeOS checkout, starting with a branch of repo
with this change. Verified that the kernel projects and no others were
identified as having shared object stores, and that repo successfully disabled
automatic pruning in their configs. Re-enabled pruning and ran repo sync just
on one of the kernel directories. Verified that pruning was re-disabled as a
result.
Change-Id: I728ed5b06f0087aeb5a23ba8f5410a7cd10af5b0
The requirement to explicitly specify the local project when starting
a new repo branch is somewhat counter intuitive.
This patch uses the current directory's git tree as the default
project.
Tested by running
'repo start <name>'
observed that the result is the same as if running
'repo start <name> .'
Change-Id: If106caa801b4cd5ba70dbe8354a227d59f100aa3
When nothing is pending, most of this code is already short-circuited.
Hoist the single check up to make this more obvious/slightly faster.
Change-Id: Iec3a7e08eacd23a7c5f964900d5776bf5252c804
The constant prompting when registered hooks change can be tedious and
has a large multiplication factor when the project is large (e.g. the
AOSP). It gets worse as people want to write more checks, hooks, docs,
and tests (or fix bugs), but every CL that goes in will trigger a new
prompt to approve.
Let's tweak our trust model when it comes to hooks. Since people start
off by calling `repo init` with a URL to a manifest, and that manifest
defines all the hooks, anchor trust in that. This requires that we get
the manifest over a trusted link (e.g. https or ssh) so that it can't
be MITM-ed. If the user chooses to use an untrusted link (e.g. git or
http), then we'll fallback to the existing hash based approval.
Bug: Issue 226
Change-Id: I77be9e4397383f264fcdaefb582e345ea4069a13
Make it possible to exclude projects using regex/wildcard.
The syntax is similar to that of the -r option, e.g.:
repo forall -i ^platform/ ^device/ -c 'echo $REPO_PROJECT'
Change-Id: Id250de5665152228c044c79337d3ac15b5696484
It was only displaying 'Project list error: GitError()'
without any useful info about the project nor the error
Change-Id: Iad66cbaa03cad1053b5ae9ecc90d7772aa42ac13
As soon as we wrote the gitc manifest, the folder for that repo became
empty, causing the next GetProjects lookup to fail. Reorder the
GetProjects calls so that they all happen while we still have the
repository contents available.
If you were already in a subdir, for cases like 'repo start <branch> .',
this would still fail, since the working directory would disappear out
from under you. That's fine most of the time, since we shouldn't be
doing operations based on the local directory, but git has a realpath
function that tries to restore CWD by chdir'ing back to it. So if the
working directory no longer exists, chdir to the topdir before
continuing.
Change-Id: Ibdf6cd37ff6e5a5f8338347c3919175491f7166f
Some teams have a continuous build server that would mark certain
manifest green and safe to sync to. Then team members could repo
sync to that particular manifest file and make sure they always
sync to a green build. But if she/he has some local changes and
wants to rebase, currently it would be a manual process to find the
correct version to rebase onto. This patch helps with that use
case by automating the process to rebase onto the currently synced
manifest version.
Change-Id: I847c9eb6addf7f84fd3f5594fbf8c0bcc103f9a5
When repo sync is used with -f (--force-error) and a project fails to
sync, the sync will continue but then exit with an error status.
However if -n (--network-only) is also used, the exit code is 0, even
when a project failed.
Modify the logic to make sure the sync exits with the correct status.
Bug: Issue 214
Change-Id: I0b5d97a34642c5aa3743750ef14a42c9d5743c1d
By passing --prune to the sync command, the --prune option is
given to the `git fetch`, causing refs that no longer exist on
the remote to be removed.
Change-Id: I3cedacce14276d96ac2d5aabf2d07fd05e92bc02
Adds windows as one of the allowed platforms flags.
Fixes -p foo to append 'platform-foo', instead of each letter (list.extend
expects a list and thus appends each char in the string, rather than the
string itself).
Change-Id: I73a92127ac29a32fc31b335cc54a246302904140
With gitc-init, a gitc client may be specified using '-c'. If we're
not currently in that client, we need to change directories so that
we don't affect the local checkout, and to ensure that repo is
checked out in the new client.
This also makes '-c' optional if already in a gitc client, to match
the rest of the init options.
Change-Id: Ib514ad9fd101698060ae89bb035499800897e9bd
The .gitcookies file generated by googlesource.com does not have
the header:
# (Netscape) HTTP Cookie File
which causes python's MozillaCookieJar.load to fail with the
error:
"does not look like a Netscape format cookies file"
Prepend the expected header onto the generated cookie file.
We don't bother to check if the header already exists on the
file; repeating it does not cause any problem.
Bug: Issue 207
Change-Id: I7d39720a1d36a6aae00f70691156514ebc04e579
repo gitc-delete deletes a GITC client and all the locally
saved sources. Useful for removing unnecessary clients and
recovering disk space.
Change-Id: Idf23addcea52b8713d268c34a7b37da0c5e5cd26
These won't show up as common commands in the help text unless in a GITC
client, and will refuse to execute.
Change-Id: Iffe82adcc9d6ddde9cb4b204f83ff018042bdab0
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.
This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.
Change-Id: I435235cb5622a048ffad0059affd32ecf71f1f5b
This way any changes made to the main manifest are reflected in the gitc
manifest. It's also necessary to use both manifests to sync since the
information required to update the gitc manifest is actually in the repo
manifest.
This also fixes a few issues that came up when testing. notdefault
groups weren't being saved to the gitc manifest in a method that matched
'sync'. The merge branch wasn't always being set to the correct value
either.
Change-Id: I5dbc850dd73a9fbd10ab2470ae4c40e46ff894de
Updates the repo launcher and gitc_utils to pull the manifest
directory location out of the gitc config file.
Change-Id: Id08381b8a7d61962093d5cddcb3ff6afbb13004b
The GITC filesystem does not understand relative URLs for remotes,
so now if a remote uses a relative URL, it will be be expanded to
be relative to the manifest URL.
Change-Id: Ie1210758560aeb1934da3f71496aaf19c2728214
Add repo start support for GITC checkouts. If the user is in
the GITC FS view, they can now run repo start to check out
the sources and create a new working branch.
When "repo start" is called on a GITC project, the revision
tag is set to an empty string and saved in a new tag:
old-revision. This tells the GITC filesystem to display the
local copy of the sources when being viewed. The local copy
is created by pulling the project sources and the new branch
is created based off the original project revision.
Updated main.py to setup each command's gitc_manifest when
appropriate.
Updated repo sync's logic to sync opened projects and
updating the GITC manifest file for the rest.
Change-Id: I7e4809d1c4fc43c69b26f2f1bebe45aab0cae628
Don't emit a message when the netrc file doesn't exist or couldn't
be opened.
Instead of trying to unpack the result of info.authenticators() and
catching the resulting TypeError when it's None, first store it to
a local and only unpack it if it has a value.
Also remove an unused import.
Change-Id: I5c404d91e48c261c1ab850c3e5f040c4f4c235cb
Use the same cookies and proxy that git traffic goes through for
persistent-http[s] to support authentication for smart-sync.
Change-Id: I20f4a281c259053a5a4fdbc48b1bca48e781c692
Add repo sync support for GITC checkouts. If the user is in the
GITC client directory they can still pull the sources as normal
if they pass in the --force-gitc argument. Otherwise the user
should call repo sync in the GITC view to update the user's
remote view. (This works because .repo in the GITC view will
link to .repo in the client config directory.)
Part of the support for this change is the refactoring of GITC
related code into gitc_utils.py.
Change-Id: I2636aaa50b450b6f091309db8dd0e8f4dbdad579
Adds the new gitc-init command to set up a GITC client. Gitc-init
sets up the client directory and calls repo init within it. Once
the repo is initialized, then generates a GITC manifest file
by using git ls-remote on each project and retrieving the HEAD SHA
to use as the revision attribute.
Gitc-init inherits from and has all the options as repo init.
Change-Id: Icd7e47e90eab752a77de7c80ebc98cfe16bf6de3
Previously repo would only print the failing project path if
Sync_NetworkHalf returned false/empty, but if it threw an
exception the print() was never called.
Change-Id: I58c41de43930df5e34b21561c205e062a72e290f
Enable operating against groups of repositories. As it stands, it isn't
compatible with `-r/--regex`.
`repo forall -g groupname -c pwd` will run `pwd` for all projects in
groupname.
`repo forall -g thisgroup,-butnotthisone -c pwd` will run `pwd` for all
projects in `thisgroup` but not `butnotthisone`.
`repo list -g groupname -n` will list all the names of repos in
`groupname`.
Change-Id: Ia75c50ce52541d1c8cea2874b20a4db2e0e54960
In some cases, a user may wish to continue with a sync even though
it would require overwriting an existing git directory. This behavior
is not safe as a default because it could result in the loss of some
user data, but as an optional flag it allows the user more flexibility.
To support this, add a --force-sync flag to the sync command that will
attempt to overwrite the existing git dir if it is specified and the
existing git dir points to the wrong obj dir.
Change-Id: Ieddda8ad54e264a1eb4a9d54881dd6ebc8a03833
repo info will crash when using a manifest with no default element despite
default being an optional element. Output nothing for "Manifest Branch" if no
default element exists (or if no default revision exists).
Change-Id: I7ebffa2408863837ba980f0ab6e593134400aea9
If a workspace is synced with the -s or -t option, the included projects
may be different to those in the original manifest. However, when using
the forall command, the list of the projects from the original manifest
is used.
If the smart sync manifest file exists, use it to override the original
manifest.
Change-Id: Iaefcbe148d2158ac046f158d98bbd8b5a5378ce7
When syncing with the -s or -t option, a smart_sync_override.xml file
is created. This file is left in the file system when syncing again
without the -s or -t option.
Remove the smart sync override manifest, if it exists, when not using
the -s or -t option.
Change-Id: I697a0f6405205ba5f84a4d470becf7cd23c07b4b
git rev-parse fails for projects that don't have an explicit revision
specified, and don't have a branch of the same name as the default
revision. This can be the case in a workspace synced with the smart
sync (-s) or smart tag (-t) option.
Change-Id: I19bfe9fe7396170379415d85f10f6440dc6ea08f
The error message only states that writing the manifest failed.
Include the exception message, so it's easier to track down the reason
that the write failed.
Change-Id: I06e942c48a19521ba45292199519dd0a8bdb1de7
After performing the actual cherry-pick operation, the code
in cherry_pick.py opens a pipe to 'git commit -F' to rewrite the commit
message, emits the fixed-up commit msg to the pipe, then waits
for 'git commit' to complete. The child 'git' process winds up
hanging while reading from the pipe, however, since the parent
process still has it open. To fix the hang, change the parent process
to close its end of the pipe after it has emitted the message.
Change-Id: I5929371e69a5b076f09009d00d40a2c72ac8ac33
If the generator that produces per-project worker arguments raises an
exception it triggers python bug http://bugs.python.org/issue8296.
Rewrite the generator expression as a generator function, and catch
Exceptions and KeyboardInterrupts to end the iteration.
Also add a pool worker initializer to disable SIGINT to prevent
KeyboardInterrupts inside multiprocessing.Pool in the worker threads
causing the same problem.
Fixes easy-to-reproduce hangs when hitting ctrl-c during
repo forall -c echo
Change-Id: Ie4a65b3e1e07a64ed6bb6ff20f3912c4326718ca
In 2fb6466f79 an optimisation was
added to avoid fetching from remotes if the project is fixed to
a revision and the revision is already available locally.
This causes problems for users who expect all objects to be
fetched by default.
Change the logic so that the optimized behaviour is only enabled if
an option is explicitly given to repo sync.
Change-Id: I3b2794ddd8e0071b1787e166463cd8347ca9e24f
Before, a list was generated, which is why there was a massive delay.
Using a generator will allow processes to start straight away
Change-Id: Ia325b0b340cc328c08c9bcc92a6709bbdaf6a664
buflist was being used, which isn't available in Python 3.
`Execute` was using StringIO to capture the output of `PrintWorkTreeStatus`,
only to redirect it straight to stdout.
Instead, just let `PrintWorkTreeStatus` do it's own thing directly to stdout.
for handling `_FindOrphans`, we swap StringIO for a list. Nothing was done
that needed a a file like object.
Change-Id: Ibdaae137904de66a5ffb590d84203ef0fe782d8b
This reverts commit 38e4387f8e.
A "repo init" followed by "repo sync" is meant to be as safe as
"git clone". In particular it should not run arbitrary code provided
by the manifest owner.
It would still be nice to have support for manifest-defined git hooks
--- they'd just need a prompt like the upload RepoHook has. Hopefully
a later change can bring them back.
Change-Id: I5ecd90fb5c2ed64f103d856d1ffcba38a47b062d
Signed-off-by: Jonathan Nieder <jrn@google.com>
When working within a team or corporation it is often
useful/required to use predefined git templates. This
change teaches repo to use a per-remote git hook template
structure.
The implementation is done as a continuation of the
existing projecthook functionality. The terminology is
therefore defined as projecthooks.
The downloaded projecthooks are stored in the .repo
directory as a metaproject separating them from the users
project forest.
The projecthooks are downloaded and set up when doing a
repo init and updated for each new repo init.
When downloading a mirror the projecthooks gits are
not added to the bare forest since the intention is to
ensure that the latest are used (allows for company policy
enforcement).
The projecthooks are defined in the manifest file in the
remote element as a subnode, the name refers to the
project name on the server referred to in the remote.
<remote name="myremote ..>
<projecthook name="myprojecthookgit" revision="myrevision"/>
</remote>
The hooks found in the projecthook revision supersede
the stock hooks found in repo. This removes the need for
updating the projecthook gits for repo stock hook changes.
Change-Id: I6796b7b0342c1f83c35f4b3e46782581b069a561
Signed-off-by: Patrik Ryd <patrik.ryd@stericsson.com>
Signed-off-by: Ian Kumlien <ian.kumlien@gmail.com>
When there are uncommitted files in the tree, 'repo upload' stops to
ask if it is OK to continue, but does not report the actual names of
uncommitted files.
This patch adds plumbing to have the outstanding file names reported
if desired.
BUG=None
TEST=verified that 'repo upload' properly operates with the following
conditions present in the tree:
. file(s) modified locally
. file(s) added to index, but not committed
. files not known to git
. no modified files (the upload proceeds as expected)
Change-Id: If65d5f8e8bcb3300c16d85dc5d7017758545f80d
Signed-off-by: Vadim Bendebury <vbendeb@chromium.org>
Signed-off-by: Vadim Bendebury <vbendeb@google.com>
When current is "split" (i.e. some projects are current while others are not):
- Disable 'not in' printout (i.e. will print out all projects)
- Disable printing of multiple projects on one line
- Print current projects in green, non-current in white
Since using color to differentiate current from non-current in "split" cases:
- In non-split cases also print out project names in color (green for current
white for non-current)
Change-Id: Ia6b826612c708447cecfe5954dc767f7b2ea2ea7
Enable '--jobs' ('-j') option in the forall subcommand. For -jn
where n > 1, the '-p' option can no longer guarantee the
continuity of console output between the project header and the
output from the worker process.
SIG_INT is sent to all worker processes upon keyboard interrupt
(Ctrl+C).
Bug: Issue 105
Change-Id: If09afa2ed639d481ede64f28b641dc80d0b89a5c
Use JSON as it is shown to be much faster than pickle.
Also clean up the loading and saving functions.
Change-Id: I45b3dee7b4d59a1c0e0d38d4a83b543ac5839390
iterator.next() was replaced with iterator.__next__() in Python 3.
Use next(iterator) instead which will select the correct method for
returning the next item.
Change-Id: I6d0c89c8b32e817e5897fe87332933dacf22027b
For long-running forall commands sometimes it's useful to know which
iteration is currently running. Add REPO_I and REPO_COUNT environment
variables to reflect the current iteration count as well as the total
number of iterations so that the user can build simple status
indicators.
Example:
$ repo forall -c 'echo $REPO_I / $REPO_COUNT; git gc'
1 / 579
Counting objects: 41, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (19/19), done.
Writing objects: 100% (41/41), done.
Total 41 (delta 21), reused 41 (delta 21)
2 / 579
Counting objects: 53410, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (10423/10423), done.
Writing objects: 100% (53410/53410), done.
Total 53410 (delta 42513), reused 53410 (delta 42513)
3 / 579
...
Change-Id: I9f28b0d8b7debe423eed3b4bc1198b23e40c0c50
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
The `review.URL.autocopy` setting sends email notification to the
named reviewers, but does not add them as reviewer on the uploaded
change.
Add a new setting `review.URL.autoreviewer`. The named reviewers
will be added as reviewer on the uploaded change.
Change-Id: I3fddfb49edf346f8724fe15b84be8c39d43e7e65
Signed-off-by: bijia <bijia@xiaomi.com>
This command allows a deeper diff between two manifest projects.
In addition to changed projects, it displays the logs of the
commits between both revisions for each project.
Change-Id: I86d30602cfbc654f8c84db2be5d8a30cb90f1398
Signed-off-by: Julien Campergue <julien.campergue@parrot.com>
The fetch logic is now shared between the jobs == 1 and
jobs > 1 cases. This refactoring also fixes a bug where
opts.force_broken was not honored when jobs > 1.
Change-Id: Ic886f3c3c00f3d8fc73a65366328fed3c44dc3be
Currently if you run repo download -c on a change and the cherry-pick
runs into a merge conflict a Traceback is produced:
rob@rob-i5-lm ~/Programming/repo_test/repo1 $ repo download -c repo1 3/1
From ssh://rob-i5-lm:29418/repo1
* branch refs/changes/03/3/1 -> FETCH_HEAD
error: could not apply 0c8b474... 2
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'
Traceback (most recent call last):
File "/home/rob/Programming/git-repo/main.py", line 408, in <module>
_Main(sys.argv[1:])
File "/home/rob/Programming/git-repo/main.py", line 384, in _Main
result = repo._Run(argv) or 0
File "/home/rob/Programming/git-repo/main.py", line 143, in _Run
result = cmd.Execute(copts, cargs)
File "/home/rob/Programming/git-repo/subcmds/download.py", line 90, in Execute
project._CherryPick(dl.commit)
File "/home/rob/Programming/git-repo/project.py", line 1943, in _CherryPick
raise GitError('%s cherry-pick %s ' % (self.name, rev))
error.GitError: repo1 cherry-pick 0c8b4740f876f8f8372bbaed430f02b6ba8b1898
This amount of error message is confusing to users and has the side effect
of the git message telling you the actual issue being ignored.
This change introduces a message stating that the cherry-pick couldn't
be completed removing the Traceback.
To reproduce the issue create a change that causes a conflict with one currently
in review and use repo download -c to cherry-pick the conflicting change.
Change-Id: I8ddf4e0c8ad9bd04b1af5360313f67cc053f7d6a
This takes the wrapper importing code from main.py and moves it into
its own module so that other modules may import it without causing
circular imports with main.py.
Change-Id: I9402950573933ed6f14ce0bfb600f74f32727705
the value of Manifest.projects has changed from being the dictionary
to the values of the dictionary. Here we handle this change
correctly on a PostRepoUpgrade.
From a `git diff v1.12.7 -- manifest_xml.py`:
+ @property
def projects(self):
self._Load()
- return self._projects
+ return self._paths.values()
self._paths does contain the projects according to this line of
manifest_xml.py:
484 self._paths[project.relpath] = project
Change-Id: I141f8d5468ee10dfb08f99ba434004a307fed810
The backtrace currently occurs when one uses the "--cbr" argument with
the repo upload subcommand if the current branch is not tracking an
upstream branch. There may be other cases that would backtrace as well,
but this is the only one I found so far.
Change-Id: Ie712fbb0ce3e7fe3b72769fca89cc4c0e3d2fce0
This significantly reduces sync time and used brandwidth as only
a tar of each project's revision is checked out, but git is not
accessible from projects anymore.
This is relevant when git is not needed in projects but sync
speed/brandwidth may be important like on CI servers when building
several versions from scratch regularly for example.
Archive is not supported over http/https.
Change-Id: I48c3c7de2cd5a1faec33e295fcdafbc7807d0e4d
Signed-off-by: Julien Campergue <julien.campergue@parrot.com>
* Add .decode('utf-8') where needed
* Add 'b' to `open` where needed, and remove where unnecessary
Change-Id: I0f03ecf9ed1a78e3b2f15f9469deb9aaab698657
git-repo uses 2 space indentation. A couple of recent changes
introduced 4 space indentation in some modules.
Change-Id: Ia4250157c1824c1b5e7d555068c4608f995be9da
It is often useful to be able to include the same project more than
once, but with different branches and placed in different paths in the
workspace. Add this feature.
This CL adds the concept of an object directory. The object directory
stores objects that can be shared amongst several working trees. For
newly synced repositories, we set up the git repo now to share its
objects with an object repo.
Each worktree for a given repo shares objects, but has an independent
set of references and branches. This ensures that repo only has to
update the objects once; however the references for each worktree are
updated separately. Storing the references separately is needed to
ensure that commits to a branch on one worktree will not change the
HEAD commits of the others.
One nice side effect of sharing objects between different worktrees is
that you can easily cherry-pick changes between the two worktrees
without needing to fetch them.
Bug: Issue 141
Change-Id: I5e2f4e1a7abb56f9d3f310fa6fd0c17019330ecd
Example:
- `repo init -b master` / sync a project
- In one project: `git checkout -b work origin/branch-thats-not-master`
- make some changes, `git commit`
- `repo upload .`
- Upload will now be skipped with a warning instead of being uploaded to
master
Change-Id: I990b36217b75fe3c8b4d776e7fefa1c7d9ab7282
Passing a project revisionExpr to UploadForReview will cause it to
try to push to refs/for/<sha> if the revision points to a sha
instead of a branch. Pass None for dest_branch if no destination
branch has been specified, which will cause UploadForReview to
upload to the merge branch.
There is room for further improvement, the user prompts will
still print "Upload project <project> to remote branch <sha>",
and then upload to the merge branch and not the sha, but that
is the same behavior that was in 1.12.2.
Change-Id: I06c510336ae67ff7e68b5b69e929693179d15c0b
When the RPC call fails, the error message returned by the server
is printed, but it is not obvious that this is caused by RPC call
failure.
Prefix the error message with a descriptive message that explains
what went wrong.
Change-Id: I4b77af22aacc2e9843c4df9d06bf54e41d9692ff
When syncing using smart sync or smart tag mode, print the url of
the manifest server that is being used.
This is useful in organisations that have multiple manifest servers
used in different manifest branches.
Change-Id: Ib5bc2de5af6f4a942d0ef735c65cbc0721059a61
The command `repo upload --cbr -D <some branch>` will display
the default revision, and not the actual destination branch.
Fix that and display the branch to which the change will be
uploaded to.
Change-Id: I712ed0871c819dce6774c47254dac4efec4532e0
* manifest_name was never set if opt.smart_sync or opt.smart_tag is used.
* Set it earlier, so that the code handles it correctly when it is None.
* An UnboundLocalError is raised if running `repo sync` without any options:
local variable 'manifest_name' referenced before assignment
* This fixes the above regression caused by commit
53a6c5d93a
Change-Id: I57086670f3589beea8461ce0344f6ec47ab85b7b
Revert "Fix "'module' object is not callable" error", and fix it properly.
* The urlparse module is renamed to urllib.parse in Python 3.
* This commit fixes the code to use "urllib.parse.urlparse"
instead of creating a new module urlib and setting
urlib.parse to urlparse.urlparse.
* Fixes an AttributeError:
'function' object has no attribute 'uses_relative'
This reverts commit cd51f17c64.
Change-Id: I48490b20ecd19cf5a6edd835506ea5a467d556ac
In a couple of files the urlparse method was not being set up
correctly for python < 3 and this resulted in an error being
thrown when trying to call it.
Change-Id: I4d2040ac77101e4e228ee225862f365ae3d96cec
This was broken in b2bd91c, which updated the manifest after it had
been overridden, which made it fall back to the original file (and
not the one from the manifest server).
This builds on 0766900 and overrides the manifest by the one
downloaded from the manifest server completely.
Change-Id: Ic3972390a68919b614616631d99c9e7a63c0e0db
It doesn't make sense to print the relpath, since there's nothing
checked out there and the dir shouldn't even exist.
Change-Id: Id43631a8e0895929d3a5ad4ca8c2dc9e3d233e70
This fixes dest-branch display with >1 branch being uploaded to at
once, and correctly handles setting the target branch in that case.
Change-Id: If5e9c7ece02cc0d903e2cb377485ebea73a07107
This adds the ability to have reviews pushed to a different branch
from the one on which changes are based. This is useful for "gateway"
systems without smartsync.
Change-Id: I3a8a0fabcaf6055e62d3fb55f89c944e2f81569f
Add a new module with methods for checking the Python version.
Instead of handling Python3 imports with try...except blocks, first
check the python version and then import the relevant modules. This
makes the code a bit cleaner and will result in less diff when/if we
remove support for Python < 3 later.
Use the same mechanism to handle `input` vs. `raw_input` and add
suppression of pylint warnings caused by redefinition of the built-in
method `input`.
Change-Id: Ia403e525b88d77640a741ac50382146e7d635924
Also-by: Chirayu Desai <cdesai@cyanogenmod.org>
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
When running 'repo init --reference=<mirror>', the mirror will be
used for all projects except the manifest project. This is because
the _InitGitDir function uses the 'repo.reference' git config
value specified in the manifest git, which has no effect when
creating the manifest git as that value will be set after the git
has been successfully cloned.
Information about where the manifest git is located on the server
is only known when performing the 'repo init', so that information
has to be provided when cloning the git in order for it to set up
a proper mapping.
Change-Id: I47a2c8b3267a4065965058718ce1def4ecb34d5a
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
Filter the project list based on regex or wildcard matching
of strings, then we can handle subset of all projects.
Change-Id: Ib6c23aec79e7d981f7b6a5eb0ae93c44effec467
Signed-off-by: Zhiguang Li <muzili@gmail.com>
* Fix imports.
* Use python3 syntax.
* Wrap map() calls with list().
* Use list() only wherever needed.
(Thanks Conley!)
* Fix dictionary iteration methods
(s/iteritems/items/).
* Make use of sorted() in appropriate places
* Use iterators directly in the loop.
* Don't use .keys() wherever it isn't needed.
* Use sys.maxsize instead of sys.maxint
TODO:
* Make repo work fully with python3. :)
Some of this was done by the '2to3' tool [1], by
applying the needed fixes in a way that doesn't
break compatibility with python2.
Links:
[1]: http://docs.python.org/2/library/2to3.html
Change-Id: Ibdf3bf9a530d716db905733cb9bfef83a48820f7
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
* Print project name if the "quiet" option is not used.
Change-Id: I99863bb50f66e4dcbaf2d170bdd05971f2a4e19a
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
`repo list -n` prints only the name of the projects.
`repo list -p` prints only the path of the projects.
Change-Id: If7d78eb2651f0b1b2fe555dc286bd2bdcad0d56d
Signed-off-by: Chirayu Desai <cdesai@cyanogenmod.org>
Change Details:
* Make "default" a special manifest group that matches any project that
does not have the special project group "notdefault"
* Use "default" instead of "all,-notdefault" when user does not specify
manifest group
* Expand -g option help to include example usage of manifest groups
Change Benefits:
* Allow a more intuitive and expressive manifest groups specification:
* "default" instead of "all,-notdefault"
* "default,foo" instead of "all,-notdefault,foo"
* "default,-foo" instead of "all,-notdefault,-foo"
* "foo,-default" which has no equivalent
* Default manifest groups behavior can be restored by the command
'repo init -g default'. This is significantly more intuitive than the
current equivalent command 'repo init -g all,-notdefault'.
Change-Id: I6d0673791d64a650110a917c248bcebb23b279d3
List the user's manifest groups when running `repo info`.
These groups are passed to `repo init` using the -g/--groups flag.
Change-Id: Ie8a4ed74a35b8b90df3b1ee198fe725b1cd68ae7
Several messages are printed with the `print` method and the message
is split across two lines, i.e.:
print('This is a message split'
'across two source code lines')
Which causes the message to be printed as:
This is a message splitacross two source code lines
Add a space at the end of the first line before the line break:
print('This is a message split '
'across two source code lines'
Also correct a minor spelling mistake.
Change-Id: Ib98d93fcfb98d78f48025fcc428b6661380cff79
Add an option to pass `--no-tags' to `git fetch'.
Change-Id: I4158cc369773e08e55a167091c38ca304a197587
Signed-off-by: Mitchel Humpherys <mitchelh@codeaurora.org>
`repo cherry-pick` was broken because we were referencing stderr
instead of sys.stderr. This should fix it.
Change-Id: I67f25c3a0790d029edc65732c319df7c684546c8
If a workspace is initialised with:
repo init -u git://path/to/manifest -b manifest-branch
and the default.xml specifies the default revision as `other-branch`,
running `repo info -d` results in a GitError:
fatal: bad revision 'refs/remotes/m/other-branch..'
The repo info command uses the default revision to build the symlink
to the remote revision which is passed to the `git rev-list` command.
This is incorrect; the manifest's branch name should be used.
Change-Id: Ibae5b91869848276785facfaef433e38d49fd726
'repo status --orphans' searches for non-repo objects
(not within a project), which is particularly helpful
before removing a working tree.
Change-Id: I2239c12e6bc0447b0ad71129551cb50fa671961c
In the information message displayed after running repo init, there
is a missing space:
If this is not the directory in which you want to initializerepo
Add a space.
Change-Id: I20467673ba7481cfe782ba58ff6ed2f7ce9824a5
If repo init is run with the --mirror option, repo checks if there
is already a workspace initialized in the current location, and if
so, exits with an error message:
--mirror not supported on existing client
This error can cause confusion; the users do not understand what
is wrong and what they need to do to fix it.
Change the error message to make it a bit clearer.
Change-Id: Ifd06ef64fd264bd1117e4184c49afe0345b75d8c
If the current manifest is broken then "repo sync" fails because it
can't retrieve the default value for --jobs. Use 1 in this case, in
order that you can "repo sync" to get a fixed manifest (assuming someone
fixed it upstream).
Change-Id: I4262abb59311f1e851ca2a663438a7e9f796b9f6
(Previous submission of this change broke Android buildbot due to
incorrect regular expression for parsing git-config output. During
investigation, we also found that Android, which pulls Chromium, has a
workaround for Chromium's submodules; its manifest includes Chromium's
submodules. This new change, in addition to fixing the regex, also
take this type of workarounds into consideration; it adds a new
attribute that makes repo not fetch submodules unless submodules have a
project element defined in the manifest, or this attribute is
overridden by a parent project element or by the default element.)
We need a representation of git-submodule in repo; otherwise repo will
not sync submodules, and leave workspace in a broken state. Of course
this will not be a problem if all projects are owned by the owner of the
manifest file, who may simply choose not to use git-submodule in all
projects. However, this is not possible in practice because manifest
file owner is unlikely to own all upstream projects.
As git submodules are simply git repositories, it is natural to treat
them as plain repo projects that live inside a repo project. That is,
we could use recursively declared projects to denote the is-submodule
relation of git repositories.
The behavior of repo remains the same to projects that do not have a
sub-project within. As for parent projects, repo fetches them and their
sub-projects as normal projects, and then checks out subprojects at the
commit specified in parent's commit object. The sub-project is fetched
at a path relative to parent project's working directory; so the path
specified in manifest file should match that of .gitmodules file.
If a submodule is not registered in repo manifest, repo will derive its
properties from itself and its parent project, which might not always be
correct. In such cases, the subproject is called a derived subproject.
To a user, a sub-project is merely a git-submodule; so all tips of
working with a git-submodule apply here, too. For example, you should
not run `repo sync` in a parent repository if its submodule is dirty.
Change-Id: I4b8344c1b9ccad2f58ad304573133e5d52e1faef
The manifest URL and mirror location can be specified in environment
variables which will be used if the options are not passed on the
command line
Change-Id: Ida87968b4a91189822c3738f835e2631e10b847e
Add a new option (-e, --abort-on-errors) which will cause forall to
abort without iterating through remaining projects if a command
exits unsuccessfully.
Bug: Issue 17
Change-Id: Ibea405e0d98b575ad3bda719d511f6982511c19c
Signed-off-by: Victor Boivie <victor.boivie@sonyericsson.com>
The info command will print information regarding the current manifest
and local git branch. It will also show the difference of commits
between the local branch and the remote branch.
It also incorporates an overview command into info which shows commits
over all branches.
Change-Id: Iafedd978f44c84d240c010897eff58bbfbd7de71
Enable the following Pylint warnings:
C0322: Operator not preceded by a space
C0323: Operator not followed by a space
C0324: Comma not followed by a space
And make the necessary fixes.
Change-Id: I74d74283ad5138cbaf28d492b18614eb355ff9fe
The repo coding style is to indent at 2 characters, but there are
many places where this is not followed.
Enable pylint warning "W0311: Bad indentation" and make sure all
indentation is at multiples of 2 characters.
Change-Id: I68f0f64470789ce2429ab11104d15d380a63e6a8
When prompting for yes/no answers, convert the answer to lower
case before comparing. This makes it easier to catch answers
like "Yes", "yes", and "YES" with a comparison only for "yes".
Change-Id: I06da8281cec81a7438ebb46ddaf3344d12abe1eb
The --manifest-server-* flags broke the smartsync subcmd since
the corresponding variables weren't getting set. This change
ensures that they will always be set, regardless of whether we are
using sync -s or smartsync.
Change-Id: I1b642038787f2114fa812ecbc15c64e431bbb829
In Python3, range() creates a generator rather than a list.
None of the parameters in the ranges changed looked large enough
to create an impact in memory in Python2. Note: the only use of
range() was for iteration and did not need to be changed.
This is part of a series of changes to introduce Python3 support.
Change-Id: I50b665f9296ea160a5076c71f36a65f76e47029f
This minimum version is required for the -c argument to set config on
the command line. Without this option, git by default uses as many
threads per invocation as there are CPUs, so we cannot safely
parallelize without hosing a system.
Change-Id: I8fd313dd84917658162b5134b2d9aa34a96f2772
Fixing some more pylint warnings:
W1401: Anomalous backslash in string
W0623: Redefining name 'name' from outer scope
W0702: No exception type(s) specified
E0102: name: function already defined line n
Change-Id: I5afcdb4771ce210390a79981937806e30900a93c
If the parent of current directory has an initialized repo,
for example, if the current directory is
'/home/users/harry/platform/ics', and there is an initialized repo
in harry's home directory '/home/users/harry/.repo', when user
run 'repo init' command, repo is always initialized to parent
directory in '/home/users/harry/.repo', but most of time user
intends to initialize repo in the current directory, this patch
tells user how to do it.
Change-Id: Id7a76fb18ec0af243432c29605140d60f3de85ca
Previously, if a key was added, a client wouldn't add the key during
the sync step. This would cause issues if a new key were added and a
subsequent release were signed by that key.
Change-Id: I4fac317573cd9d0e8da62aa42e00faf08bfeb26c
We can't just let this run wild with a high (or even low) -j, since
that would hose a system. Instead, limit the total number of threads
across all git gc subprocesses to the number of CPUs reported by the
multiprocessing module (available in Python 2.6 and above).
Change-Id: Icca0161a1e6116ffa5f7cfc6f5faecda510a7fb9
The repo list -r command will execute a regex search for every
argument provided on both the project name and the project
worktree path.
Useful for finding rarely used gits.
Change-Id: Iaff90dd36c240b3d5d74817d11469be22d77ae03
Try to more accurately estimate which projects take the longest to
sync by keeping an exponentially weighted moving average (a=0.5) of
fetch times, rather than just recording the last observation. This
should discount individual outliers (e.g. an unusually large project
update) and hopefully allow truly slow repos to bubble to the top.
Change-Id: I72b2508cb1266e8a19cf15b616d8a7fc08098cb3
Some projects may consistently take longer to fetch than others, for
example a more active project may have many more Gerrit changes than a
less active project, which take longer to transfer. Use a simple
heuristic based on the last fetch time to fetch slower projects first,
so we do not tend to spend the end of the sync fetching a small number
of outliers.
This algorithm is probably not optimal, and due to inter-run latency
variance and Python thread scheduling, we may not even have good
estimates of a project sync time.
Change-Id: I9a463f214b3ed742e4d807c42925b62cb8b1745b
"except Exception as e" instead of "except Exception, e"
This is part of a transition to supporting Python 3. Python >= 2.6
support "as" syntax.
Note: this removes Python 2.5 support.
Change-Id: I309599f3981bba2b46111c43102bee38ff132803
pylint configuration file (.pylintrc) is added, and submission
instructions are updated to include pylint usage steps.
Deprecated pylint suppression (`disable-msg`) is updated in a few
modules to make it work properly with the latest version (0.26).
Change-Id: I4ec2ef318e23557a374ecdbf40fe12645766830c
We need a representation of git-submodule in repo; otherwise repo will
not sync submodules, and leave workspace in a broken state. Of course
this will not be a problem if all projects are owned by the owner of the
manifest file, who may simply choose not to use git-submodule in all
projects. However, this is not possible in practice because manifest
file owner is unlikely to own all upstream projects.
As git submodules are simply git repositories, it is natural to treat
them as plain repo projects that live inside a repo project. That is,
we could use recursively declared projects to denote the is-submodule
relation of git repositories.
The behavior of repo remains the same to projects that do not have a
sub-project within. As for parent projects, repo fetches them and their
sub-projects as normal projects, and then checks out subprojects at the
commit specified in parent's commit object. The sub-project is fetched
at a path relative to parent project's working directory; so the path
specified in manifest file should match that of .gitmodules file.
If a submodule is not registered in repo manifest, repo will derive its
properties from itself and its parent project, which might not always be
correct. In such cases, the subproject is called a derived subproject.
To a user, a sub-project is merely a git-submodule; so all tips of
working with a git-submodule apply here, too. For example, you should
not run `repo sync` in a parent repository if its submodule is dirty.
Change-Id: I541e9e2ac1a70304272dbe09724572aa1004eb5c
Fixing more issues found with pylint. Some that were supposed to
have been fixed in the previous sweep (Ie0db839e) but were missed:
C0321: More than one statement on a single line
W0622: Redefining built-in 'name'
And some more:
W0631: Using possibly undefined loop variable 'name'
W0223: Method 'name' is abstract in class 'name' but is not overridden
W0231: __init__ method from base class 'name' is not called
Change-Id: Ie119183708609d6279e973057a385fde864230c3
Fix the following issues reported by pylint:
C0321: More than one statement on a single line
W0622: Redefining built-in 'name'
W0612: Unused variable 'name'
W0613: Unused argument 'name'
W0102: Dangerous default value 'value' as argument
W0105: String statement has no effect
Also fixed a few cases of inconsistent indentation.
Change-Id: Ie0db839e7c57d576cff12d8c055fe87030d00744
Change 9bb1816b removed part of a block of code, but left the
remaining part unreachable. Remove it.
Change-Id: Icdc6061d00e6027df32dee9a3bad3999fe7cdcbc
Currently when doing a sync against a revision locked manifest,
sync has no option but to fall back to sync'ing the entire refs space;
it doesn't know which ref to ask for that contains the sha1 it wants.
This sucks if we're in -c mode; thus when we generate a revision
locked manifest, record the originating branch- and try syncing that
branch first. If the sha1 is found within that branch, this saves
us having to pull down the rest of the repo- a potentially heavy
saving.
If that branch doesn't have the desired sha1, we fallback to sync'ing
everything.
Change-Id: I99a5e44fa1d792dfcada76956a2363187df94cf1
Add two new command line options, -u/--manifest-server-username and
-p/--manifest-server-password, which can be used to specify a username
and password to authenticate to the manifest server when using the
-s/--smart-sync or -t/--smart-tag option.
If -u and -p are not specified when using the -s or -t option, use
authentication credentials from the .netrc file (if there are any).
Authentication credentials from -u/-p or .netrc are not used if the
manifest server specified in the manifest file already includes
credentials.
Change-Id: I6cf9540d28f6cef64c5694e8928cfe367a71d28d
manifest_xml: import `HEAD` and `R_HEADS` from correct module
version: import `HEAD` from correct module
`HEAD` and `R_HEADS` should be imported from the git_refs module,
where they are originally defined, rather than from the project
module.
repo: remove unused import of readline
cherry_pick: import standard modules on separate lines
smartsync: import subcmd modules explicitly from subcmd
Use:
`import re
import sys`
and
`from subcmds.sync import Sync`
Instead of:
`import sys, re`
and
`from sync import Sync`
Change-Id: Ie10dd6832710939634c4f5c86b9ba5a9cd6fc92e
