Arasphere was unable to resolve auth checks for old arasphere events

[ara4n]

Whilst investigating #1733 I came across this. One of the federation requests (from darmstadt) that was failing, spending hours spidering for missing events (including auth events), and then failed the event entirely because it couldn't find the m.room.create... despite it being in matrix.org's DB.

2016-12-30 14:01:48,336 - synapse.handlers.federation - 1506 - INFO - PUT-12409 - Different auth: set(['$141661037721nBtSl:arasphere.net', '$1416450173215crCca:matrix.org'])
2016-12-30 14:01:48,337 - synapse.state - 385 - INFO - PUT-12409 - Resolving state for !cURbafjkfsMDVwdRDQ:matrix.org with 2 groups
2016-12-30 14:01:48,337 - synapse.handlers.federation - 1550 - INFO - PUT-12409 - Different auth after resolution: set(['$141661037721nBtSl:arasphere.net'])
2016-12-30 14:01:48,338 - synapse.federation.federation_server - 154 - ERROR -  - Failed to handle PDU
Traceback (most recent call last):
  File "synapse/federation/federation_server.py", line 147, in on_incoming_transaction
    )
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "synapse/federation/federation_server.py", line 578, in _handle_new_pdu
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "synapse/federation/federation_server.py", line 612, in _handle_new_pdu
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "synapse/handlers/federation.py", line 174, in on_receive_pdu
    origin,
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "synapse/handlers/federation.py", line 1173, in _handle_new_event
    (event_map[e_id].type, event_map[e_id].state_key): event_map[e_id]
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "synapse/handlers/federation.py", line 1351, in _prep_event
    try:
  File "/home/matrix/.synapse/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "synapse/handlers/federation.py", line 1637, in do_auth
    "rejects": {
  File "synapse/api/auth.py", line 130, in check
    logger.info(
SynapseError: 403: Room '!cURbafjkfsMDVwdRDQ:matrix.org' does not exist

Full logs on ~/synapse/darmstadt.log on arasphere.

[richvdh]

unlikely we're ever going to investigate this now. Probably fixed, who even knows.