[7.x] [kbn/optimizer] share all plugin bundles (#68986) (#69234)

* [kbn/optimizer] share all plugin bundles (#68986)

Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* [BundleRefPlugin] resolve imports to files too

(cherry picked from commit 2bb6ac1a796c6b8661d9ca6c33bef9e23d7a0764)

* [kbn/optimizer] fix windows compatibility (#69304)

Co-authored-by: spalger <spalger@users.noreply.github.com>
(cherry picked from commit db1df7b)

* [BundleRefPlugin] resolve imports to files too (#69241)

Co-authored-by: spalger <spalger@users.noreply.github.com>
(cherry picked from commit e03f242)

* update extraPublicDirs to include latest changes

Co-authored-by: spalger <spalger@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Author: 3 people

.eslintrc.js

@@ -476,6 +476,7 @@ module.exports = {
     {
       files: [
         'test/functional/services/lib/web_element_wrapper/scroll_into_view_if_necessary.js',
+        'src/legacy/ui/ui_render/bootstrap/kbn_bundles_loader_source.js',
         '**/browser_exec_scripts/**/*.js',
       ],
       rules: {

NOTICE.txt

@@ -21,6 +21,11 @@ used. Logarithmic ticks are places at powers of ten and at half those
 values if there are not to many ticks already (e.g. [1, 5, 10, 50, 100]).
 For details,  see https://github.com/flot/flot/pull/1328
 
+---
+This module was heavily inspired by the externals plugin that ships with webpack@97d58d31
+MIT License http://www.opensource.org/licenses/mit-license.php
+Author Tobias Koppers @sokra
+
 ---
 This product has relied on ASTExplorer that is licensed under MIT.
 

docs/development/core/server/kibana-plugin-core-server.pluginmanifest.extrapublicdirs.md

@@ -0,0 +1,17 @@
+<!-- Do not edit this file. It is automatically generated by API Documenter. -->
+
+[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [PluginManifest](./kibana-plugin-core-server.pluginmanifest.md) &gt; [extraPublicDirs](./kibana-plugin-core-server.pluginmanifest.extrapublicdirs.md)
+
+## PluginManifest.extraPublicDirs property
+
+> Warning: This API is now obsolete.
+> 
+> 
+
+Specifies directory names that can be imported by other ui-plugins built using the same instance of the @<!-- -->kbn/optimizer. A temporary measure we plan to replace with better mechanisms for sharing static code between plugins
+
+<b>Signature:</b>
+
+```typescript
+readonly extraPublicDirs?: string[];
+```

docs/development/core/server/kibana-plugin-core-server.pluginmanifest.md

@@ -21,6 +21,7 @@ Should never be used in code outside of Core but is exported for documentation p
 |  Property | Type | Description |
 |  --- | --- | --- |
 |  [configPath](./kibana-plugin-core-server.pluginmanifest.configpath.md) | <code>ConfigPath</code> | Root [configuration path](./kibana-plugin-core-server.configpath.md) used by the plugin, defaults to "id" in snake\_case format. |
+|  [extraPublicDirs](./kibana-plugin-core-server.pluginmanifest.extrapublicdirs.md) | <code>string[]</code> | Specifies directory names that can be imported by other ui-plugins built using the same instance of the @<!-- -->kbn/optimizer. A temporary measure we plan to replace with better mechanisms for sharing static code between plugins |
 |  [id](./kibana-plugin-core-server.pluginmanifest.id.md) | <code>PluginName</code> | Identifier of the plugin. Must be a string in camelCase. Part of a plugin public contract. Other plugins leverage it to access plugin API, navigate to the plugin, etc. |
 |  [kibanaVersion](./kibana-plugin-core-server.pluginmanifest.kibanaversion.md) | <code>string</code> | The version of Kibana the plugin is compatible with, defaults to "version". |
 |  [optionalPlugins](./kibana-plugin-core-server.pluginmanifest.optionalplugins.md) | <code>readonly PluginName[]</code> | An optional list of the other plugins that if installed and enabled \*\*may be\*\* leveraged by this plugin for some additional functionality but otherwise are not required for this plugin to work properly. |

examples/demo_search/kibana.json

@@ -5,5 +5,6 @@
   "server": true,
   "ui": true,
   "requiredPlugins": ["data"],
-  "optionalPlugins": []
+  "optionalPlugins": [],
+  "extraPublicDirs": ["common"]
 }

examples/embeddable_examples/kibana.json

@@ -5,5 +5,6 @@
   "server": true,
   "ui": true,
   "requiredPlugins": ["embeddable"],
-  "optionalPlugins": []
+  "optionalPlugins": [],
+  "extraPublicDirs": ["public/todo", "public/hello_world", "public/todo/todo_ref_embeddable"]
 }

examples/url_generators_examples/kibana.json

@@ -5,5 +5,6 @@
   "server": false,
   "ui": true,
   "requiredPlugins": ["share"],
-  "optionalPlugins": []
+  "optionalPlugins": [],
+  "extraPublicDirs": ["public/url_generator"]
 }

packages/kbn-optimizer/README.md

@@ -30,6 +30,18 @@ Bundles built by the the optimizer include a cache file which describes the info
 
 When a bundle is determined to be up-to-date a worker is not started for the bundle. If running the optimizer with the `--dev/--watch` flag, then all the files referenced by cached bundles are watched for changes. Once a change is detected in any of the files referenced by the built bundle a worker is started. If a file is changed that is referenced by several bundles then workers will be started for each bundle, combining workers together to respect the worker limit.
 
+## Bundle Refs
+
+In order to dramatically reduce the size of our bundles, and the time it takes to build them, bundles will "ref" other bundles being built at the same time. When the optimizer starts it creates a list of "refs" that could be had from the list of bundles being built. Each worker uses that list to determine which import statements in a bundle should be replaced with a runtime reference to the output of another bundle.
+
+At runtime the bundles share a set of entry points via the `__kbnBundles__` global. By default a plugin shares `public` so that other code can use relative imports to access that directory. To expose additional directories they must be listed in the plugin's kibana.json "extraPublicDirs" field. The directories listed there will **also** be exported from the plugins bundle so that any other plugin can import that directory. "common" is commonly in the list of "extraPublicDirs".
+
+> NOTE: We plan to replace the `extraPublicDirs` functionality soon with better mechanisms for statically sharing code between bundles.
+
+When a directory is listed in the "extraPublicDirs" it will always be included in the bundle so that other plugins have access to it. The worker building the bundle has no way of knowing whether another plugin is using the directory, so be careful of adding test code or unnecessary directories to that list.
+
+Any import in a bundle which resolves into another bundles "context" directory, ie `src/plugins/*`, must map explicitly to a "public dir" exported by that plugin. If the resolved import is not in the list of public dirs an error will be thrown and the optimizer will fail to build that bundle until the error is fixed.
+
 ## API
 
 To run the optimizer from code, you can import the [`OptimizerConfig`][OptimizerConfig] class and [`runOptimizer`][Optimizer] function. Create an [`OptimizerConfig`][OptimizerConfig] instance by calling it's static `create()` method with some options, then pass it to the [`runOptimizer`][Optimizer] function. `runOptimizer()` returns an observable of update objects, which are summaries of the optimizer state plus an optional `event` property which describes the internal events occuring and may be of use. You can use the [`logOptimizerState()`][LogOptimizerState] helper to write the relevant bits of state to a tooling log or checkout it's implementation to see how the internal events like [`WorkerStdio`][ObserveWorker] and [`WorkerStarted`][ObserveWorker] are used.

packages/kbn-optimizer/package.json

@@ -45,6 +45,7 @@
     "terser-webpack-plugin": "^2.1.2",
     "tinymath": "1.2.1",
     "url-loader": "^2.2.0",
+    "val-loader": "^1.1.1",
     "watchpack": "^1.6.0",
     "webpack": "^4.41.5",
     "webpack-merge": "^4.2.2"

packages/kbn-optimizer/src/__fixtures__/mock_repo/plugins/bar/public/index.ts

@@ -19,6 +19,6 @@
 
 import './legacy/styles.scss';
 import './index.scss';
-import { fooLibFn } from '../../foo/public/index';
+import { fooLibFn } from '../../foo/public';
 export * from './lib';
 export { fooLibFn };