JupyterLab regression

[krinsman]

JupyterLab expects the memory usage information from NBResuse (i.e. what's waiting for it at the <base_url>/metrics endpoint) to have the JSON format specified by the following interface:

  export interface IMetricRequestResult {
    rss: number;
    limits: {
      memory?: {
        rss: number;
        warn?: number;
      };
    };
  }

However, after the update to use Prometheus (0.3.4+), the JSON response emitted "by NBResuse" (i.e. at the <base_url>/metrics endpoint) is now of the form as Prometheus's HTTP API, see here. So JupyterLab can't read it, so thinks NBResuse isn't installed, and thus no longer displays memory usage information even when NBResuse is installed.

There seems to be a fairly clear mapping between the arguments passed to the constructors of the Prometheus Gauge's here in the NBResuse code and the response of the metadata portion of the Prometheus HTTP API, but I'm not sure about the format of the HTTP response encoding the actual values of the metrics. That knowledge seems to be embedded implicitly in the Javascript code for the NBExtension, but I still feel I need to look into it more closely to be sure.

Since this is a feature regression, I think it should be a priority to find some way to fix this as soon as possible, although it will probably require making a PR to JupyterLab. If that's the case, then we should probably hold off on making any such PR before stabilizing the "API" for NBResuse.

[GlennHD]

@krinsman , you appear to know a lot more about this than I do... I'm running JupyterHub and had the same issue. It was a bit frustrating that it didn't work at first and I ran a bunch of things to no avail last night... I logged back into Jupyterhub today and the memory is showing up fine. I wish I remembered exactly what I did but I didn't take good notes. I just remember doing the below at one point and rebooting the machine today sometime. Hope this leads to some solution... good luck.

sudo -E pip3 install nbresuse[resources]
sudo -E pip3 install psutil

[jtpio]

It sounds like it would be better to wait a bit for the new API (Prometheus response) to stabilize before making the changes in JupyterLab.

Maybe it's possible to deprecate the 0.3.4 version on PyPI? Since it's a breaking change, bumping to 0.4.0 or even 1.0.0 would give the different frontends some time and security to update to newer versions of nbresuse.

As a side note, the conda forge bot didn't pick up the new release and hence didn't create a new PR on the feedstock: https://github.com/conda-forge/nbresuse-feedstock

In the meantime, users can pin nbresuse to 0.3.3 so it works in JupyterLab (and with the system-monitor extension).

[jtpio]

Also looking briefly at the code for the classic notebook extension, it looks like the warning (red background color) is not displayed anymore?

[jtpio]

Maybe it's possible to deprecate the 0.3.4 version on PyPI?

This is referred to as "yanking" on PyPI:

[jtpio]

@krinsman should we proceed with the yanking of 0.3.4 and re-release the Prometheus changes as 0.4.0?

[jtpio]

Also I wonder if the original API should be kept alongside the prometheus response.

Working with a JSON response sounds more natural than parsing the plain text Prometheus metrics with regexes. It might also facilitate the eventual switch to EventSource (#19).