This repository was archived by the owner on Apr 26, 2024. It is now read-only.
-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
This repository was archived by the owner on Apr 26, 2024. It is now read-only.
SerializationFailure
when rotating push actions #13117
Copy link
Copy link
Closed
Labels
S-MajorMajor functionality / product severely impaired, no satisfactory workaround.Major functionality / product severely impaired, no satisfactory workaround.T-DefectBugs, crashes, hangs, security vulnerabilities, or other reported issues.Bugs, crashes, hangs, security vulnerabilities, or other reported issues.X-RegressionSomething broke which worked on a previous releaseSomething broke which worked on a previous releaseX-Release-BlockerMust be resolved before making a releaseMust be resolved before making a release
Description
We recently changed things so we rotate notifications almost immediately, rather than waiting a day. This interacts badly with people sending read receipts, as that transaction attempts to modify the push summary tables.
We need to fix this or revert before we make another release.
Broke by #13005
Stack trace:
SerializationFailure: could not serialize access due to concurrent update
File "synapse/metrics/background_process_metrics.py", line 240, in run
return await func(*args, **kwargs)
File "synapse/storage/databases/main/event_push_actions.py", line 806, in _rotate_notifs
caught_up = await self.db_pool.runInteraction(
File "synapse/storage/database.py", line 835, in runInteraction
return await delay_cancellation(_runInteraction())
File "twisted/internet/defer.py", line 1656, in _inlineCallbacks
result = current_context.run(
File "twisted/python/failure.py", line 514, in throwExceptionIntoGenerator
return g.throw(self.type, self.value, self.tb)
File "synapse/storage/database.py", line 807, in _runInteraction
result = await self.runWithConnection(
File "synapse/storage/database.py", line 930, in runWithConnection
return await make_deferred_yieldable(
File "twisted/python/threadpool.py", line 244, in inContext
result = inContext.theWork() # type: ignore[attr-defined]
File "twisted/python/threadpool.py", line 260, in <lambda>
inContext.theWork = lambda: context.call( # type: ignore[attr-defined]
File "twisted/python/context.py", line 117, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "twisted/python/context.py", line 82, in callWithContext
return func(*args, **kw)
File "twisted/enterprise/adbapi.py", line 282, in _runWithConnection
result = func(conn, *args, **kw)
File "synapse/storage/database.py", line 923, in inner_func
return func(db_conn, *args, **kwargs)
File "synapse/storage/database.py", line 671, in new_transaction
r = func(cursor, *args, **kwargs)
File "synapse/storage/databases/main/event_push_actions.py", line 850, in _rotate_notifs_txn
self._rotate_notifs_before_txn(txn, rotate_to_stream_ordering)
File "synapse/storage/databases/main/event_push_actions.py", line 950, in _rotate_notifs_before_txn
txn.execute_batch(
File "synapse/storage/database.py", line 329, in execute_batch
self._do_execute(
File "synapse/storage/database.py", line 395, in _do_execute
return func(sql, *args, **kwargs)
File "synapse/storage/database.py", line 330, in <lambda>
lambda the_sql: execute_batch(self.txn, the_sql, args), sql
File "psycopg2/extras.py", line 1187, in execute_batch
cur.execute(b";".join(sqls))
Sentry: https://sentry.tools.element.io/organizations/element/issues/924/
Metadata
Metadata
Assignees
Labels
S-MajorMajor functionality / product severely impaired, no satisfactory workaround.Major functionality / product severely impaired, no satisfactory workaround.T-DefectBugs, crashes, hangs, security vulnerabilities, or other reported issues.Bugs, crashes, hangs, security vulnerabilities, or other reported issues.X-RegressionSomething broke which worked on a previous releaseSomething broke which worked on a previous releaseX-Release-BlockerMust be resolved before making a releaseMust be resolved before making a release