Commit Graph

5 Commits

Author SHA1 Message Date
Mike Frysinger
13f323b2c2 event_log: turn id generation from a generator to a func call
Running lots of sync processes in parallel can hit the failure:
Fetching projects:  23% (124/523)Exception in thread Thread-201:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/src/repo/subcmds/sync.py", line 278, in _FetchProjectList
    success = self._FetchHelper(opt, project, *args, **kwargs)
  File "/usr/local/src/repo/subcmds/sync.py", line 357, in _FetchHelper
    start, finish, success)
  File "/usr/local/src/repo/event_log.py", line 104, in AddSync
    event = self.Add(project.relpath, task_name, start, finish, success)
  File "/usr/local/src/repo/event_log.py", line 74, in Add
    'id': (kind, next(self._next_id)),
ValueError: generator already executing

It looks like, while we lock the multiprocessing value correctly, the
generator that wraps the value isn't parallel safe.  Since we don't
have a way of doing that (as it's part of the language), turn it into
a plain function call instead.

Bug: https://crbug.com/gerrit/10293
Change-Id: I0db03601986ca0370a1699bab32adb03e7b2910a
2019-01-14 16:11:08 -05:00
Eli Ribble
d4b13c280b Leverage the next keyword from python 2.7
This is literally what the next keyword is for.
https://www.python.org/dev/peps/pep-3114/

Change-Id: I843755910b847737b077ff2361ba3e04409db0f0
2018-12-19 11:06:35 -08:00
Dylan Deng
e469a0c741 fix some sync error while using python3
Change-Id: I70925e48756c356d48359679d8ad1b9e33a68595
2018-07-24 22:20:08 +08:00
David Pursehouse
685320b000 event_log: Fix order of parameters to Add method call
Change-Id: I5add20eadfde39806ef4b2cc819da0ae0bfec2f5
2018-01-10 11:05:14 +09:00
David Riley
e0684addee sync: Add support to dump a JSON event log of all sync events.
Change-Id: Id4852968ac1b2bf0093007cf2e5ca951ddab8b3b
2017-05-29 13:39:54 +09:00