Make the pubsub system more generic

[snowteamer]

Summary of changes:

• Pubsub clients can now subscribe to any "channel" represented by a distinct string ID, not just contracts.
  For future flexibility, only the most basic kind of channel is implemented for now (outside Chelonia contracts, which only use pubsub to broadcast GIMessage objects). In particular there are no Channel objects.
• Clients can publish data to any channel using pub(channelID, data). The server has a corresponding PUB message handler, which by default just broadcasts the message to any subscriber.

[taoeffect]

Nicely done @snowteamer! I've assigned @corrideat to review while I'm away. I might be able to do a more in-depth review later today before my break starts (we'll see). But I did get a chance to at least run this and found a flow error:

Running "exec:flow" (exec) task
Error ----------------------------------------------------------------------------------------- backend/pubsub.js:262:50

Unexpected token `=`

   262|       server.subscribersByChannelID[channelID] ??= new Set()
                                                         ^

[taoeffect]

Also noticed this failing test when I ran it with grunt test (headless):

group-chat-scrolling.spec.js.mp4

But... it passed the second time when I ran it with grunt dev and npm run cy:open 🤔

EDIT: currently @snowteamer and I think this has nothing to do with the changes in this PR.

[taoeffect]

Note: 44d73ae#r1423320892

[corrideat]

Because of how /sync works now, calling /isSyncing first is unnecessary (it should be a no-op in this case). Also, since /sync is asynchronous, there should be a .catch at the end to handle errors gracefully.

[snowteamer]

Thanks!

[taoeffect]

@snowteamer I remember vaguely that there was an event that caused a subscribe to happen when a reconnection was established on the websocket (after say the internet was out for a bit). Is this that event?

Because if so we need to force it to fetch the latest events with { force: true } I think...

@snowteamer & @corrideat can you investigate this question?

[snowteamer]

@taoeffect That sounds like the reconnection-succeeded event, which causes pending sub actions to be retried. Therefore many subscriptions may be triggered upon reconnection.
I thought fetching latest events was the default behavior of sync.

[taoeffect]

@snowteamer sync now takes an extra parameter (an options object). Have a look at how it's implemented. The reason force was added was because /sync is called a lot of times in various places.

[snowteamer]

@taoeffect So that was added to avoid unnecessary network requests, e.g. to avoid checking the contract's latestHash every time /sync is called? Ok, that makes sense.

But I'm not sure why a plain /sync call couldn't automatically bail out without network calls if the system knows that the given contract is already fully synced.

In my understanding, a contract is already synced unless:

• it is new, i.e. we have no log entries for it yet ;
• it is marked as currently syncing ;
• we just logged in or reconnected after a disconnection, hence we might have missed new log entries in the meantime which should be fetched.

[SebinSong]

I'm no expert in this area so mostly focused on looking for any syntax related issues, but didn't find any.
Great work. @snowteamer

[taoeffect]

Question

[taoeffect]

Why was this file created? And it doesn't seem to be used anywhere...? If not used, and no reason for it, plz delete

[snowteamer]

It was created to avoid repeating the [pubsub] tag in every console call and to remove the following block in backend/pubsub.js:

const log = console.log.bind(console, tag)
log.bold = (...args) => console.log(bold(tag, ...args))
log.debug = console.debug.bind(console, tag)
log.error = (...args) => console.error(bold.red(tag, ...args))

but it also needs bold and color support, so it's not ready for use yet

[taoeffect]

Approved! Great work @snowteamer!