need advice for a long blocking task

[florianschanda]

Hi,

We're implementing a new vscode extension for a DSL. In some cases the parsing and checking can take quite a while.

Right now we're doing a parse on each TEXT_DOCUMENT_DID_CHANGE, but if the parse and validation take ~10 seconds then events queue up and the error highlighting lags.

How should we deal with this? Is there any way to inspect the queue? I'd like to look at the queue and simply skip all events except the last one (so that we always eventually get a correctly highlighted source).

[perrinjerome]

In the scenario that you describe, when the events queue up and the error highlighting lags, if you enable logging in your server code, do you also observe this log ?

pygls/pygls/protocol.py

Line 303 in de7a33e

logger.warning('Cancel notification for unknown message id "%s"', msg_id)

We discussed a bit in the past ( #240 (comment) ) and we said that it might be a possible missing feature (or bug maybe) of pygls that it does not process the cancel notifications.

[florianschanda]

This message does not appear.

[florianschanda]

I also tried to do something with multiprocessing, but there is some state that I need to maintain and I got really weird behaviour. I assume the thread you start are all done via multiprocessing, so any state that needs maintaining needs to also be done with multiprocessing shared memory? Or are the threads you launch for that threading.Thread?

Is there a non-trivial example somewhere that makes use of the pygls thread() decorator?

[alcarney]

Is there a non-trivial example somewhere that makes use of the pygls thread() decorator?

There is an example of the thread decorator being used in the example json_server.py, if that helps

[alcarney]

I'd like to look at the queue and simply skip all events except the last one (so that we always eventually get a correctly highlighted source).

I'm not sure if this is exactly what you want, but here is a pattern that I've been experimenting with recently

class SphinxManager(LanguageFeature):
    """Responsible for managing Sphinx application instances."""

    def __init__(self, client_factory: SphinxClientFactory, *args, **kwargs):
 
        self._pending_builds: Dict[str, asyncio.Task] = {}
        """Holds tasks that will trigger a build after a given delay if not cancelled."""

    async def document_change(self, params: lsp.DidChangeTextDocumentParams):

        # Cancel any existing pending builds
        if (task := self._pending_builds.pop(client.id, None)) is not None:
            task.cancel()

        self._pending_builds[client.id] = asyncio.create_task(
            self.trigger_build_after(uri, client.id, delay=2)
        )

    async def trigger_build_after(self, uri: Uri, app_id: str, delay: float):
        """Trigger a build for the given uri after the given delay."""
        await asyncio.sleep(delay)

        self._pending_builds.pop(app_id)
        await self.trigger_build(uri)

I've omitted a lot from the code above to try and focus on the main idea, the full code can be found here if you are interested.

• The document_change() method is called on each TEXT_DOCUMENT_DID_CHANGE notification.
  It checks in self._pending_builds to see if there is an existing task, cancelling it if finds one.

• Then asyncio.create_task() creates a new task that will trigger a new build - assuming the given delay has passed without it being cancelled.

• The actual build started by self.trigger_build() is handled by a subprocess, so the main server is free to do other things while the build is happening

It seems to work well for my use case, where a user can make modifications to a document with the server only refreshing after a short pause in activity.