Pluggable URL Service (Direct Access Link Service)

[stacey-gammon]

Direct Access Link Service

Goal

Provide an pluggable API that generates direct access links for applications with state.

As a bonus, conditionally store it as a short url with an optional expiration attribute.

Why is this important?

We can still more forward with many features without this service, so why should we prioritize it?

Reason 1: Migrations for URLs in saved objects

Without a migration plan, we can end up with a lot of URLs stored in various saved objects that are broken and deprecated. We hit this issue with reporting - Watcher configurations had a lot of legacy style reporting URLs. Even though we could deprecate those URLs in a major, we chose not to for awhile because it would be so disruptive to our users - breaking all of there watches. Now we have multiple features in the midst of development that will also potentially require storing URL references. We'll end up with the same issues but even more widespread.

Consider the situation with threshold alerts, wanting a reference to the URL of wherever the embeddable resided. One workaround without this service is taking whatever the current URL is and storing that with the alert, but that will result in us having no idea of what URLs are being stored and referenced and what URL changes would cause a BWC - no tests around it.

Reason 2: Clear division of responsibilities

Rather than having hacked together URLs handled by multiple teams, it should be the responsibility of the application that is rendering in that URL.

Considerations

Server side functionality

We will likely want at least the URL generation services to be available server side for certain alerts (for example, you can imagine wanting a link to a dashboard that you sent to "run in background" (#53335) , or a threshold alert that you created based off of a visualization in a dashboard (#49908).

Since I think we should rely on our new URL state sync utilities, I think that means we should move some of that functionality into common folder and ensure we do not directly rely on client side specific functionality (e.g. defaulting a variable to window.location.href).

Migrations

Some features may end up storing references to urls via this system, by storing a generatorId and state. Which team should be in charge of migrating these saved objects if the state stored has been deprecated? I believe we can solve the problem by having generators supply a migrateState function. Then the feature authors managing that specific saved object will be in charge of noticing deprecated url state stored, looping through the saved objects and running migrateState in it to store the new state.

API

GET api/generate_access_link 
{
  // A unique identifier for a particular URL. Plugins will use this for handling migrations for different URLs. For instance you could have a generatorId of `dashboardApp`, or `siemOverview`.  Each application or plugin can register many generator ids.
  generatorId: string,
  // The State Management Service will use this to restore the URL. It will be much easier
  // having this stored as key: state (e.g. `{ '_a' : { dashboardId: 123 } }` for migrations than
  // as a single url string.
  state: { [key: keyId]: Object },
  options?: {
      // Optionally create and return the link as a permalink that will be stored in elasticsearch
      // and less susceptible to breaking changes in the URL structure. The
      // benefit of a short url is that they can be BWC. Rather than simply store a mapping
      // of a short url to a link, we should store all the information required to generate
      // the link so we can run it through the generation process at load up time, to give us
      // a place to convert from one version to another seamlessly.
      permalink?: boolean,
      // If permalink is true, the following options are required:
      permalinkOptions?: {
        version: string,
        // Indicate when this short url should be automatically deleted from the database.
        // This could be a future implementation, not necessary in phase one, and would
        // be reliant on the task manager as a periodic clean up task would be implemented on
        // top of it.  Format also TBD. Could use the moment strings, such a "1w, 1d", or accept
        // a timestamp.
        expires?: string,
        // Optionally allow the user to specify a name when creating the short url rather than
        // an auto generated id.
        // TODO: I recall some possible security issues with doing this, need to sync with
        // Court or security team to make sure I am not forgetting something...
        id?: string
      }
  }
}

Plugin

export class DirectAccessLinkPlugin
  implements Plugin<DirectAccessLinkSetup, DirectAccessLinkStart> {
  private generators: { [key: string]: DirectAccessLinkGenerator<typeof key> } = {};

  constructor(initializerContext: PluginInitializerContext) {}

  public setup(core: CoreSetup) {
    return {
      registerLinkGenerator: (generator: DirectAccessLinkGenerator<string>) => {
        this.generators[generator.id] = generator;
      },
    };
  }

  public start(core: CoreStart) {
    return {
      generateUrl: <G extends GeneratorId>(id: G, state: GeneratorStateMapping[G]) => {
        const generator = this.generators[id] as DirectAccessLinkGenerator<typeof id>;
        if (!generator) {
          throw new Error(`No link generator exists with id ${id}`);
        }
        return generator.generateUrl(state);
      },
    };
  }

  public stop() {}
}

Storage

When the Generate Access Link service is used with permalink: true then a saved object of type permalink will
be created in the .kibana index with all the information needed to recreate the non-permalink version of the link.

Management

Current issues with our short url service:

Short urls never get deleted from es
We plan to store these new short url access links as saved objects which means we can easily
expose them in the saved object management interface for management capabilities, such as delete, with little additional effort.

With the capability of detecting parent-child object relationships, broken reference links could be
identified.

Additional feature requests

Human readable links
Solved via the permalinkOptions.id attribute.

How this works with Embeddables

Many actions that are tied to embeddables would like to use URLs. For instance, if a dashboard wanted to store all it's queries in a background saved object, and alerts. In each case, there is a desire to store a URL that contains the specific embeddable, but the dashboard doesn't know where it came from. We may be able to pass in an optional object to Embeddables to supply this information to actions based on where they are embedded.

Migrations

implementors of the system can handle migrations "on the fly", using the version string, or register a saved object migration.

Registries

Server/Common:

  DirectAccessLink.registerURLGenerator<T extends { [key: string]: Object }>({
    generatorId: string,
    toUrl: (state: T) => string,
    toState: (url: string) => T,
  });

Routes:

  registerRoute('api/url_generator/to_url', {
    const { generatorId, state } = request.params;
    return DirectAccessLink.getUrlGenerator(generatorId).toUrl(state);
});

// Not sure we need this one or just the "to URL" version.
registerRoute('api/url_generator/to_state', {
    const { generatorId, url } = request.params;
    return DirectAccessLink.getUrlGenerator(generatorId).toState(url);
});

Embeddable threshold alert integration

Threshold alerts integration could use this service like so:

createThresholdAlert(sourceEmbeddable, alertParameters, alertAction) {
  const previewUrl = directAccessLinkPlugin.getLinkGenerator('embeddableReportingViewer').createLink(sourceEmbeddable.getInput());
  alertAction.attachImageUrl(previewUrl);
}

Chat ops integration

A chat ops integration could use this service like:

route.add('api/chatops', (params) => {
  const { command, ...args } = parse(params);
  if (command === 'dashboard') {
    const { dashboardId, filters, query, timeRange } = args;

    // Use the direct access link generator to create a link for the dashboard but in preview mode.
    const previewUrl = directAccessLinkPlugin.getLinkGenerator('dashboardReportPreview').createLink({
      dashboardId, filters, query, timeRange
    });
    const png = await ReportingPlugin.generatePng(previewUrl);

    // Use the direct access link generator to create a link for the main dashboard app page.
    const mainUrl = directAccessLinkPlugin.getLinkGenerator('dashboardApp').createLink({
      dashboardId, filters, query, timeRange
    });

    return { message, image: png, link: mainUrl};
  
  }
}); 

Make it slow integration

Background searches could use this integration as well, if we passed along a linkGenerator to the embeddable to get access to the url/state of the context it exists in, so when we click "view searches" we can send the user back to the original URL, but also ensure that it's set to reference the search ids in the collector cache

[elasticmachine]

Pinging @elastic/kibana-platform

[elasticmachine]

Pinging @elastic/kibana-app

[jasonrhodes]

This looks interesting but I don't fully understand it and I'm wondering if it's at all related to the linking use-case I'm thinking of? I'll describe that here for comment. :)

When APM wants to link to ML, or Infra to SIEM, etc etc, the app that wants to render a link (source app) needs to consider a few things:

• Does the license give access to that link destination?
• Does the current space have access to that link destination?
• Is that link destination enabled in Kibana at all?
• What is the url format for the link destination (and what happens if the target app changes that URL format later?)

In observability we've suggested one solution to this problem could be having each plugin define a set of "linking functions" that provide link hrefs for anything other apps may want to link to import { linkToTrace } from 'links/apm' etc. Required params would be outlined via Typescript, so the linkToTrace function may require a traceId, etc. That link will return either a) a string href, or b) some indication that the destination is not accessible, which can be handled in the source app.

If apps use their own link functions to generate links within their own app, they would be guaranteed to stay accurate even if paths, query params, etc. change.

Again, not sure if this problem is related to this issue here, but this is the problem we see most often right now.

[stacey-gammon]

Thanks for the input Jason, I do think it's related and should be thought about when something like this is designed. It's not just links to saved objects but links to application pages, and they might not be backed by a saved object. So my original proposal which was written up a good bit ago should probably be rethought. But I agree it's an issue I keep see coming up, and will again when we want the user to be able to customize drilldown links.

[stacey-gammon]

After thinking about this, it probably makes more sense to start with a client side only solution, which is easier and I think would still cover most of the use cases. Ignore the short url use cases for an initial phase.

I've started thinking the action framework might work well. Something like:

const navigateAction = uiActions.get(DASHBOARD_NAVIGATION_ACTION);
const disabled = navigateAction.isCompatible();
return <Link onClick={navigateAction.execute({ query: this.state.dynamicQuery, filters: this.state.dynamicFilters })} disabled={disabled} />;

Attach this action to the "APPLY_FILTER_TRIGGER" and drilldown links work.

Been meaning to try to come up with a POC for this, but haven't gotten around to it yet.