Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to leave a certrain room (KeyError). #9260

Open
Kimiblock opened this issue Dec 18, 2023 · 3 comments
Open

Unable to leave a certrain room (KeyError). #9260

Kimiblock opened this issue Dec 18, 2023 · 3 comments

Comments

@Kimiblock
Copy link

Description

My account on the Synapse instance can't leave #offtopic:archlinux.org. Element said MatrixError: [500] Internal server error.

Steps to reproduce

  • Trying to leave #offtopic:archlinux.org on any client
  • Receive the error

Homeserver

moechat.kimiblock.top

Synapse Version

1.98.0rc1

Installation Method

Other (please mention below)

Database

Running a single instance of PostgreSQL

Workers

Single process

Platform

  • OS: Arch Linux
  • Synapse [Self Packaged]: Using PKGBUILD from here and replaced pkgver and _tag for 1.98 rc1
  • PostgreSQL: 16.1-1
  • Python: 3.11.6-1
  • Architecture: x86_64
  • Processor: Ryzen 7 5700U

Configuration

mscs:

experimental_features:
  msc3773_enabled: true
  msc3391_enabled: true
  msc3890_enabled: true
  msc3266_enabled: true
  msc4028_enabled: true
  msc3026_enabled: true

Sliding Sync Proxy do exist, message retention set to:

retention:
    enabled: true
    default_policy:
        min_lifetime: 60d
        max_lifetime: 90d
    allowed_lifetime_min: 90d
    allowed_lifetime_max: 180d
    purge_jobs:
      - longest_max_lifetime: 3d
        interval: 12h
      - shortest_max_lifetime: 3d
        interval: 1d

Connected to Mautrix Telegram bridge, but that shouldn't affect this.

Relevant log output

Dec 18 20:23:06 serverOS-kimiblock.top synapse[566873]: 2023-12-18 20:23:06,952 - synapse.access.http.60000 - 465 - INFO - PUT-2638222 - 127.0.0.1 - 60000 - {flytothe.top} Processed request: 0.026sec/0.002sec (0.004sec, 0.001sec) (0.001sec/0.009sec/2) 11B 200 "PUT /_matrix/federation/v1/send/1702902184675-203 HTTP/1.1" "Dendrite/0.13.5+719858d" [0 dbevts]
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]: 2023-12-18 20:23:07,057 - synapse.http.server - 140 - ERROR - POST-2638226 - Failed handle request via 'RoomMembershipRestServlet': <XForwardedForRequest at 0x7f4d77202b10 method='POST' uri='/_matrix/client/v3/rooms/!ReTqqPGlVRRlnqJPJx%3Aarchlinux.org/leave' clientproto='HTTP/1.1' site='60000'>
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]: Traceback (most recent call last):
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/http/server.py", line 326, in _async_render_wrapper
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     callback_return = await self._async_render(request)
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/http/server.py", line 538, in _async_render
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     callback_return = await raw_callback_return
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:                       ^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/rest/client/room.py", line 1072, in on_POST
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     return await self._do(request, requester, room_id, membership_action, None)
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/rest/client/room.py", line 1045, in _do
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     await self.room_member_handler.update_membership(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/handlers/room_member.py", line 645, in update_membership
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     result = await self.update_membership_locked(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/handlers/room_member.py", line 1104, in update_membership_locked
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     return await self._local_membership_update(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/handlers/room_member.py", line 482, in _local_membership_update
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     await self.event_creation_handler.handle_new_client_event(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/util/metrics.py", line 113, in measured_func
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     r = await func(self, *args, **kwargs)
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/handlers/message.py", line 1493, in handle_new_client_event
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     result, _ = await make_deferred_yieldable(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     result = context.run(gen.send, result)
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/handlers/message.py", line 1571, in _persist_events
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     event = await self.persist_and_notify_client_events(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/handlers/message.py", line 1912, in persist_and_notify_client_events
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     ) = await self._storage_controllers.persistence.persist_events(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/storage/controllers/persist_events.py", line 421, in persist_events
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     ret_vals = await yieldable_gather_results(enqueue, partitioned.items())
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/util/async_helpers.py", line 307, in yieldable_gather_results
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     raise dfe.subFailure.value from None
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     result = context.run(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:              ^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     return g.throw(self.type, self.value, self.tb)
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/storage/controllers/persist_events.py", line 416, in enqueue
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     return await self._event_persist_queue.add_to_queue(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/storage/controllers/persist_events.py", line 239, in add_to_queue
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     res = await make_deferred_yieldable(end_item.deferred.observe())
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/storage/controllers/persist_events.py", line 282, in handle_queue_loop
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     ret = await self._per_item_callback(room_id, item.task)
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/storage/controllers/persist_events.py", line 362, in _process_event_persist_queue_task
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     return await self._persist_event_batch(room_id, task)
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/storage/controllers/persist_events.py", line 610, in _persist_event_batch
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     ) = await self._calculate_new_forward_extremities_and_state_delta(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/storage/controllers/persist_events.py", line 690, in _calculate_new_forward_extremities_and_state_delta
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     res = await self._get_new_state_after_events(
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/storage/controllers/persist_events.py", line 876, in _get_new_state_after_events
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     old_state_groups = {
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:                        ^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:   File "/usr/lib/python3.11/site-packages/synapse/storage/controllers/persist_events.py", line 877, in <setcomp>
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     event_id_to_state_group[evid] for evid in old_latest_event_ids
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]:     ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]: KeyError: '$sX7ySz3cOPShqZYA2hMRulsUIw3-Rtgdtxv5isvUC9o'
Dec 18 20:23:07 serverOS-kimiblock.top synapse[566873]: 2023-12-18 20:23:07,061 - synapse.access.http.60000 - 465 - INFO - POST-2638226 - 127.0.0.1 - 60000 - {@kimiblockmoe:moechat.kimiblock.top} Processed request: 0.043sec/0.001sec (0.007sec, 0.003sec) (0.003sec/0.012sec/9) 55B 500 "POST /_matrix/client/v3/rooms/!ReTqqPGlVRRlnqJPJx%3Aarchlinux.org/leave HTTP/1.1" "Mozilla/5.0 (X11; Linux x86_64; rv:120.0) Gecko/20100101 Firefox/120.0" [0 dbevts]

Anything else that would be useful to know?

The latest room state in Synapse is stuck at 7 Dec 2023, 9:52:14 pm, with the source of:

{
  "content": {
    "body": "> <@fr3ax:matrix.org> oh I can now make my own pixrobots as long as I don't sell them and contribute back w my improvements?\n\nu can make ur own pixrobot's and even sell them as long as you make the source _**code**_ available to the _user_",
    "format": "org.matrix.custom.html",
    "formatted_body": "<mx-reply><blockquote><a href=\"https://matrix.to/#/!ReTqqPGlVRRlnqJPJx:archlinux.org/$lCm8ehVZeQWiId34ydIevecBev5vxQkJoAwem1Xkhlk?via=archlinux.org&via=matrix.org&via=envs.net\">In reply to</a> <a href=\"https://matrix.to/#/@fr3ax:matrix.org\">@fr3ax:matrix.org</a><br>oh I can now make my own pixrobots as long as I don&#39;t sell them and contribute back w my improvements?</blockquote></mx-reply>u can make ur own pixrobot's and even sell them as long as you make the source <em><strong>code</strong></em> available to the <em>user</em>",
    "m.mentions": {
      "user_ids": [
        "@fr3ax:matrix.org"
      ]
    },
    "m.relates_to": {
      "m.in_reply_to": {
        "event_id": "$lCm8ehVZeQWiId34ydIevecBev5vxQkJoAwem1Xkhlk"
      }
    },
    "msgtype": "m.text"
  },
  "origin_server_ts": 1701957134886,
  "sender": "@morguldir:sulian.eu",
  "type": "m.room.message",
  "unsigned": {},
  "event_id": "$V5G7ry-WY6sdUgvK9aKqQ4qgb0TWjZK7CrQNZ4uZDcs",
  "room_id": "!ReTqqPGlVRRlnqJPJx:archlinux.org"
}

Sadly I don't have logs on 7 Dec.
@schildbach
Copy link

I'm pretty sure this is a duplicate of matrix-org/synapse#13476.

@Kimiblock
Copy link
Author

Thanks for your help.

I didn’t try purging the room first though, and retention shouldn’t have kicked in because I’ve only joined that room 1 month ago.

@Galbar
Copy link

Galbar commented Jun 17, 2024

This is happening to me with room https://matrix.to/#/#unifiedpush:matrix.org . I can't send messages in it nor leave the room. Is there a workaround to this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@schildbach @Galbar @Kimiblock