An alternative to relative imports for local source code

[epixa]

The problem

Today, we import local source code with many different path formats. Client-side code can use webpack aliases, TypeScript can import from kibana, ui, or plugins (technically on either server or client), server-side JavaScript can only do relative imports.

This is a burden for us, so we agreed to fix it. Since relative imports are usable anywhere (in theory), we decided all imports should be done through relative imports. This solves the consistency problem, but it introduces others.

Relative imports across module/package/plugin boundaries inherently make those modules less portable if the system around them changes. A great example of this is the recent move of the legacy plugins, which involved updating hundreds of import paths despite the intention of those import statements remaining unchanged.

Relative imports create ambiguity with similarly named files and directories. e.g. At a glance, does import from '../../../../../../tests' import from the root level tests directory or from the tests directory inside your plugin? The deeper your code is nested, the harder that question is to answer. It's not purely visual as well, as you can't really use find+replace to identify this either.

Relative imports make many developers unhappy. There has been strong push back on relative imports for sharing code across plugins, especially from folks that do most of their development in the browser where they're used to convenient webpack aliases.

Finally, the assumption we made when adopting a relative imports strategy that they are available in all contexts is not accurate. Despite them seeming like the standard import mechanism, developers have run into issues in various places (e.g. mocks in tests) when trying to do relative imports.

Proposed solution

Code should be imported from outside the current package/system/plugin using imports from Kibana root.

Within a package/system/plugin, it's appropriate to use relative imports, though absolute imports are still acceptable when the relative import is still crazy. Most developers will do this anyway given the option.

So for example, any of these options would be OK from within a plugin called "foo":

/*
src
  plugins
    foo
      bar
        whatever.js
      test_utils
        foomocks.js
test_utils
  globalmocks.js
*/

// src/plugins/foo/bar/whatever.js

// OK
import from 'test_utils/globalmocks';
import from 'src/plugins/foo/test_utils/foomocks';
import from '../test_utils/foomocks';

// NOT OK
import from '../../../test_utils/globalmocks';

I have a proof of concept for supporting this import behavior in #40066. More work is required, like updating the webpack configuration for client-side code and getting CI passing, but overall I think this is very doable.

[elasticmachine]

Pinging @elastic/kibana-platform

[epixa]

cc @elastic/kibana-operations

[epixa]

I should mention, these sorts of imports do not really solve the portability problem, they just make the portability problem significantly easier to deal with since you can universally use find+replace for access to code within plugins

[jasonrhodes]

I'm generally +1 on anything that removes ../../ from our code. One thing to mention, the more surface area you take up on the import from 'something_magic' front, the more likely it becomes you accidentally run up against a real import named something_magic ... for that reason at my last job we picked a single root word and made all imports start with that, so if you saw an import that started with src you knew it came from our root.

We could use import from 'kibana/whatever/path' or 'kibana_root/whatever/path' or even just 'root/whatever/path' but I like the idea of having a single indication that an import is coming from a non-standard module resolution location. $0.02.

[epixa]

@jasonrhodes Unfortunately, we're limited by the implementation here. For client-side code and for typescript, we can specify any common root we want, but for server-side javascript, all we can do is specify a root path that is literally part of the file path. In other words, to support that, we'd need to move all of our source code under a common top level directory within the project. kibana/whatever/path would need to literally be $KIBANA_ROOT/kibana/whatever/path on the file system.

I think in practice we'll be OK here for Kibana's case, but it's a fair point.

Edit: The vast majority of imports in this case would be into one of three directories, src/, x-pack/, and far less frequently test/. So we're not in too bad shape.

[spalger]

I really like your suggestion @jasonrhodes.

I really wish we could solve this by just going full monorepo and making everything a package, but I realize how much work that is.

What if instead of webpack aliases and $NODE_PATH we used a babel plugin to rewrite the import paths. Then we could:

• stick with the current convention that @kbn/ packages are imported from the repo and not the registry
• limit the root-level directories that support importing, @kbn/src would work for instance but importing from the packages directory wouldn't, they would still need to be imported with @kbn/{name}
• only change the import strategies for our code, no conflicts with transient dependencies named docs or target or something
• expose this to plugins without confusing them with imports to src which don't map to their source directory
• symlink the node_modules/@kbn directories so that IDE tools that rely on node_module resolution will continue to work
• potentially unify the strategy for the server and the client, potentially rewriting ui/public imports as @kbn/ui/public or something.

I've also created a POC for this behavior here #40451

[epixa]

That’s a cool idea if we can get it to behave consistently for js/ts, server/client, and tests. I’ll check out your POC next week

[timroes]

Thanks for having that discussion. I would in general really appreciate having shorter path for the relative imports. My two cents:

I would create some @whatever synonym for the path, instead of a plain kibana/ or src/ prefix, since I like that you see that it's meant to be something special. Not sure if we could even use another non @ char (since that can still reference legal npm organizations).

Given the actual import paths, I would suggest using a different prefix than @kbn. I think using the same prefix that we use for packages might create more confusion again (besides potentially creating more issues in path resolutions in some tools, like Jest). Because @kbn/some-package would import from a package, but just the magic name @kbn/src (and potentially other ones later, as Spencer suggested) would strangely import from a completely different root, purely by the name of that second folder in the path. I see that this could potentially create us more issues and confusion later on then just using a completely different prefix, e.g. @kibana.

Given the concrete entry points I would suggest having thing that are really commonly imported from as a prefix, and not only the src level itself, since if the path might be very deep you still end up with horrible long paths. Still I would consider very carefully about adding new paths here. My suggestions would be adding something like:

• @kibana/ -> src/
• @plugins/ -> src/plugins/ (maybe even for the transition time, also src/legacy/core_plugins)
• @kbn/ -> packages/ (as it is today)

[spalger]

I like that idea @timroes, especially the part about finding a prefix that isn't a usable npm org prefix. In my POC I was able to get the following to work by editing the config.

• @src/
• $src/
• #src/
• *src/
• ()src/
• !src/

Choosing how to define the prefix is now going to be the hardest part of this 😅

[timroes]

I would vote for either $kibana (still not a huge fan of naming it src :D) or @kibana (despite the potential overlap with NPM orgas), or maybe #kibana (as the least favorite option). The others look imho simply too much 😱 to use them

[spalger]

I'm also not a fan of importing $src directly, but I think we were probably going that route because we wanted to have a root that people to go to in the source and look up files... What if we just exported $core and $legacy explicitly instead of everything in the src directory?

[timroes]

@spalger I would totally support that approach. I think we should only export places we know there is a high chance users commonly use them. So I think $core and $legacy, $legacy (and maybe $plugins separately?) is better than just offering $src.