[BUG] Failure during the running of yarn osd bootstrap

[RyanL1997]

What is the bug?
Here is the error log from GHA

warning " > ts-jest@29.1.1" has unmet peer dependency "jest@^29.0.0".
warning " > ts-jest@29.1.1" has unmet peer dependency "typescript@>=4.3 <6".
error Error: ENOENT: no such file or directory, lstat '/home/runner/work/dashboards-reporting/dashboards-reporting/OpenSearch-Dashboards/packages/osd-eslint-import-resolver-opensearch-dashboards/node_modules/resolve'
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
ERROR [bootstrap] failed:
ERROR Error: Command failed with exit code 1: /opt/hostedtoolcache/node/18.16.0/x64/lib/node_modules/yarn/bin/yarn.js install --non-interactive
          at makeError (/home/runner/work/dashboards-reporting/dashboards-reporting/OpenSearch-Dashboards/packages/osd-pm/dist/index.js:26622:11)
          at handlePromise (/home/runner/work/dashboards-reporting/dashboards-reporting/OpenSearch-Dashboards/packages/osd-pm/dist/index.js:25558:26)
          at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
          at async installInDir (/home/runner/work/dashboards-reporting/dashboards-reporting/OpenSearch-Dashboards/packages/osd-pm/dist/index.js:25298:3)
          at async Project.installDependencies (/home/runner/work/dashboards-reporting/dashboards-reporting/OpenSearch-Dashboards/packages/osd-pm/dist/index.js:[150](https://github.com/opensearch-project/dashboards-reporting/actions/runs/7228337751/job/19697602474?pr=262#step:17:151)53:5)
          at async Object.run (/home/runner/work/dashboards-reporting/dashboards-reporting/OpenSearch-Dashboards/packages/osd-pm/dist/index.js:9089:11)
          at async runCommand (/home/runner/work/dashboards-reporting/dashboards-reporting/OpenSearch-Dashboards/packages/osd-pm/dist/index.js:57802:5)
          at async Object.run (/home/runner/work/dashboards-reporting/dashboards-reporting/OpenSearch-Dashboards/packages/osd-pm/dist/index.js:257:3)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Error: Process completed with exit code 1.

During the development of the cypress CI workflow, I observed that the above error happens during the bootstrap step of OSD. However, if you retry the bootstrap command, the entire procedure will pass without a single error. (Reference to comment)

This error is constantly happening on GHA and local setup. For GHA, on PR #262, the current workaround is to add 3 retries on this command. And it is also constantly passing on its 2nd try. 😅

Reproduction of the bug?

• Install Dashboards reporting plugin into OSD and run yarn osd bootstrap

host/environment?

• main branch

Exit criteria of this issue

• Complete OSD bootstrap without any error
• Change the cypress workflow back without any retries

[ruanyl]

Having the same issue when assembling OSD distribution, yarn osd bootstrap always failed the first time with a fresh run, but succeed with another try.

[ruanyl]

This seems to break the build of opensearch-dashboards distribution with https://github.com/opensearch-project/opensearch-build, post my findings here:

In the build script, when a plugin build successfully, that plugin source code folder will then be removed. However, when getting errors, the source code folder of reportsDashboards didn't get removed.

This causes single_version_dependencies issue when used together with --continue-on-error as then multiple plugins will be pulled into the plugins folder, that messed up the dependencies among different plugins

ERROR [single_version_dependencies] Multiple version ranges for the same dependency
      were found declared across different package.json files. Please consolidate
      those to match across all package.json files. Different versions for the
      same dependency is not supported.

      If you have questions about this please reach out to the operations team.

      The conflicting dependencies are:

        cypress
          ^12.8 => reports-dashboards
          ^13.6.0 => opensearch-security-dashboards

[ruanyl]

This issue blocks functional test repo: opensearch-project/opensearch-dashboards-functional-test#985

[RyanL1997]

Hi @ruanyl, thanks for providing more findings. Yes, I think for fixing the distribution failure we had 4 PRs in all the insights repos (dashboards-observability, reporting, visualization, and query-workbench) to sync the versions of these dependencies:

• query-workbench: Sync dependencies to the latest versions dashboards-query-workbench#241
• dashboards-observability: Sync dependencies with latest versions dashboards-observability#1342
• visualization: Sync dependencies with latest versions dashboards-visualizations#310
• reporting: Sync dependencies with latest versions #268

However, for the error msg of distribution build failure, both @derek-ho [1] and @cwperks [2] also found out that there are some of the other components needs to align their dependencies.

Reference:
[1]: opensearch-project/dashboards-query-workbench#240
[2]: opensearch-project/dashboards-query-workbench#232 (comment)

[joshuali925]

i upgraded yarn.lock to latest versions following semver and bootstrapping once was enough. didn't check which one caused issues

[RyanL1997]

i upgraded yarn.lock to latest versions following semver and bootstrapping once was enough. didn't check which one caused issues

Hi @joshuali925, isn't that the yarn.lock is on latest version for now? Since I just got #268 merged.

[joshuali925]

not sure how you upgraded yarn.lock in #268? i did rm yarn.lock, somehow yarn upgrade didn't fix this

[RyanL1997]

@joshuali925 Thanks for the info! I think I updated that in the above PR, but I will double check it to see if we can safely remove the retries in our CI workflow.

[derek-ho]

@joshuali925 @RyanL1997 did you folks bootstrap with multiple plugins? I see different errors for multiple plugins, but it seems like there is also a windows failure, not sure if its related: https://build.ci.opensearch.org/blue/rest/organizations/jenkins/pipelines/distribution-build-opensearch-dashboards/runs/6871/nodes/76/steps/303/log/?start=0, either way there are a few issues here I think, not sure if it is only in this repo or not.

[RyanL1997]

For this thread I think it is just for the dashboards-reporting. And the PR #270 just fixed that. However, according to what we did to this repo, if there are multiple issues on the bootstrap, we recommended to update the yarn.lock first and to see if it fixes the issue.

[joshuali925]

@derek-ho the error log says that the cause was outdated dependency name

Error: Cannot find module '@babel/plugin-proposal-class-properties'

it should be @babel/plugin-transform-xxx, see opensearch-project/OpenSearch-Dashboards@b82aa1c#diff-caf8141894faf58ab4b0f923a3387a3acfc3de01b9f1b38c7923f40772cfc31aL41

But it should already be fixed except the dashboards-maps repo, for example in workbench it was fixed by opensearch-project/dashboards-query-workbench@b121775

Maybe should rerun build?