Fix Flaky Dashboards Visualization Tests #1187

sejli · 2024-03-28T09:24:08Z

Description

Refactor dashboards-visualization flaky tests

Issues Resolved

Check List

Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Sean Li <lnse@amazon.com>

SuZhou-Joe · 2024-03-28T09:48:54Z

cypress/integration/plugins/gantt-chart-dashboards/gantt_ui.spec.js

  beforeEach(() => {
+    CURRENT_TENANT.newTenant = 'private';


Any reason we switch to a private tenant?

Switching to private tenant since some tests are flaky due to tennant popup. For some reason, setting it to global resulted in our target index not being found.

Hmm, switched over to global but it looks like the popup still stays there on the first test Create and saves a gantt chart. @SuZhou-Joe, any ideas on why the popup still shows up even after setting the tenant, and only on the first test?

Not familiar with the tenant popup either. @kavilla Do you know how to hide the tenant popup?

cypress/integration/plugins/gantt-chart-dashboards/gantt_ui.spec.js

SuZhou-Joe · 2024-03-28T09:56:16Z

backport 2.13 needed?

Signed-off-by: Sean Li <lnse@amazon.com>

sejli · 2024-03-28T10:10:34Z

backport 2.13 needed?

Yup, need to get this backported to 2.13 so the integ tests can pass for the build.

Signed-off-by: Sean Li <lnse@amazon.com>

SuZhou-Joe · 2024-03-28T15:39:09Z

@sejli I rebase the branch but the gant cases still failed. Could you please share the result in your local env?

sejli · 2024-03-28T17:07:11Z

@sejli I rebase the branch but the gant cases still failed. Could you please share the result in your local env?

Sure, here's a the video from my latest run. I'm running with the latest 2.13 release candidates and the command I used was

➜ opensearch-dashboards-functional-test (gantt-charts-fix) ✗ yarn cypress:run-with-security --browser chrome --spec 'cypress/integration/plugins/gantt-chart-dashboards/gantt_ui.spec.js'
yarn run v1.22.21
$ env TZ=America/Los_Angeles NO_COLOR=1 cypress run --headless --env SECURITY_ENABLED=true,openSearchUrl=https://localhost:9200,WAIT_FOR_LOADER_BUFFER_MS=500 --browser chrome --spec cypress/integration/plugins/gantt-chart-dashboards/gantt_ui.spec.js

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        9.5.4                                                                          │
  │ Browser:        Chrome 123 (headless)                                                          │
  │ Node Version:   v18.16.0 (/Users/lnse/.nvm/versions/node/v18.16.0/bin/node)                    │
  │ Specs:          1 found (plugins/gantt-chart-dashboards/gantt_ui.spec.js)                      │
  │ Searched:       cypress/integration/plugins/gantt-chart-dashboards/gantt_ui.spec.js            │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  plugins/gantt-chart-dashboards/gantt_ui.spec.js                                 (1 of 1)
Browserslist: caniuse-lite is outdated. Please run:
npx browserslist@latest --update-db

Why you should do it regularly:
https://github.com/browserslist/browserslist#browsers-data-updating


  Dump test data
    ✓ Indexes test data for gantt chart

  Save a gantt chart
    ✓ Creates and saves a gantt chart (20523ms)

  Render and configure a gantt chart
    ✓ Renders no data message (6562ms)
    ✓ Renders the chart (6098ms)

  Configure panel settings
    ✓ Changes y-axis label (5716ms)
    ✓ Changes x-axis label (5102ms)
    ✓ Changes time formats (6192ms)
    ✓ Hides legends (7365ms)

  Add gantt chart to dashboard
    ✓ Adds gantt chart to dashboard


  9 passing (1m)


  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        9                                                                                │
  │ Passing:      9                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     1 minute, 2 seconds                                                              │
  │ Spec Ran:     plugins/gantt-chart-dashboards/gantt_ui.spec.js                                  │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


  (Video)

  -  Started processing:  Compressing to 32 CRF                                                     
  -  Finished processing: /Users/lnse/opensearch/opensearch-dashboards-functional-tes    (4 seconds)
                          t/cypress/videos/plugins/gantt-chart-dashboards/gantt_ui.sp               
                          ec.js.mp4                                                                 


====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  plugins/gantt-chart-dashboards/gant      01:02        9        9        -        -        - │
  │    t_ui.spec.js                                                                                │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        01:02        9        9        -        -        -  

✨  Done in 81.80s.

gantt_ui.spec.js.mp4

sejli · 2024-03-28T17:23:27Z

I switched to global tenant following PR #1073. ~~Despite setting to global tenant, I noticed that in both of our recorded videos run by GitHub (theirs for reference), the tenant icon in the top right is still the admin icon.~~ I can see that the URL includes the security tenant being set to global. However, the Creates and saves a gantt chart is still stuck on the popup.

* fixing gantt chart flaky tests Signed-off-by: Sean Li <lnse@amazon.com> * removing comments Signed-off-by: Sean Li <lnse@amazon.com> * try global tenant Signed-off-by: Sean Li <lnse@amazon.com> * try global tenant in before hook Signed-off-by: Sean Li <lnse@amazon.com> --------- Signed-off-by: Sean Li <lnse@amazon.com> (cherry picked from commit 4faa0c9)

SuZhou-Joe · 2024-03-29T00:38:38Z

It seems the tenant popup is still there.

sejli · 2024-03-29T00:43:15Z

Yes, that was the original issue I kept bringing up here and here

Hmm, switched over to global but it looks like the popup still stays there on the first test Create and saves a gantt chart. @SuZhou-Joe, any ideas on why the popup still shows up even after setting the tenant, and only on the first test?

I can see that the URL includes the security tenant being set to global. However, the Creates and saves a gantt chart is still stuck on the popup.

* fixing gantt chart flaky tests Signed-off-by: Sean Li <lnse@amazon.com> * removing comments Signed-off-by: Sean Li <lnse@amazon.com> * try global tenant Signed-off-by: Sean Li <lnse@amazon.com> * try global tenant in before hook Signed-off-by: Sean Li <lnse@amazon.com> --------- Signed-off-by: Sean Li <lnse@amazon.com> (cherry picked from commit 4faa0c9) Co-authored-by: Sean Li <lnse@amazon.com>

* fixing gantt chart flaky tests Signed-off-by: Sean Li <lnse@amazon.com> * removing comments Signed-off-by: Sean Li <lnse@amazon.com> * try global tenant Signed-off-by: Sean Li <lnse@amazon.com> * try global tenant in before hook Signed-off-by: Sean Li <lnse@amazon.com> --------- Signed-off-by: Sean Li <lnse@amazon.com>

fixing gantt chart flaky tests

5a36a3c

Signed-off-by: Sean Li <lnse@amazon.com>

sejli requested review from xluo-aws, Hailong-am, SuZhou-Joe, ruanyl, wanglam and raintygao as code owners March 28, 2024 09:24

Merge branch 'main' into gantt-charts-fix

17deb74

SuZhou-Joe reviewed Mar 28, 2024

View reviewed changes

SuZhou-Joe added the backport 2.x label Mar 28, 2024

removing comments

6314617

Signed-off-by: Sean Li <lnse@amazon.com>

sejli added 3 commits March 28, 2024 03:10

Merge branch 'main' into gantt-charts-fix

ebfd99d

try global tenant

b987d13

Signed-off-by: Sean Li <lnse@amazon.com>

try global tenant in before hook

d4eebc5

Signed-off-by: Sean Li <lnse@amazon.com>

SuZhou-Joe added v2.13.0 backport 2.13 labels Mar 28, 2024

kavilla approved these changes Mar 28, 2024

View reviewed changes

tianleh approved these changes Mar 28, 2024

View reviewed changes

SuZhou-Joe merged commit 4faa0c9 into opensearch-project:main Mar 29, 2024
37 of 38 checks passed

opensearch-trigger-bot bot mentioned this pull request Mar 29, 2024

[Backport 2.x] Fix Flaky Dashboards Visualization Tests #1190

Merged

opensearch-trigger-bot bot mentioned this pull request Mar 29, 2024

[Backport 2.13] Fix Flaky Dashboards Visualization Tests #1191

Merged

sejli mentioned this pull request Mar 29, 2024

Switch to Private Tenant for Gantt Charts Tests #1194

Closed

1 task

sejli mentioned this pull request Mar 29, 2024

[2.13] Switch to Private Tenant for Gantt Charts Tests #1195

Merged

1 task

sejli mentioned this pull request Apr 25, 2024

[ACTION NEEDED] Fix flaky integration tests at distribution level opensearch-project/dashboards-visualizations#357

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Flaky Dashboards Visualization Tests #1187

Fix Flaky Dashboards Visualization Tests #1187

sejli commented Mar 28, 2024

SuZhou-Joe Mar 28, 2024

sejli Mar 28, 2024

sejli Mar 28, 2024

SuZhou-Joe Mar 28, 2024

SuZhou-Joe commented Mar 28, 2024

sejli commented Mar 28, 2024 •

edited

Loading

SuZhou-Joe commented Mar 28, 2024

sejli commented Mar 28, 2024 •

edited

Loading

sejli commented Mar 28, 2024 •

edited

Loading

SuZhou-Joe commented Mar 29, 2024

sejli commented Mar 29, 2024 •

edited

Loading

Fix Flaky Dashboards Visualization Tests #1187

Fix Flaky Dashboards Visualization Tests #1187

Conversation

sejli commented Mar 28, 2024

Description

Issues Resolved

Check List

SuZhou-Joe Mar 28, 2024

Choose a reason for hiding this comment

sejli Mar 28, 2024

Choose a reason for hiding this comment

sejli Mar 28, 2024

Choose a reason for hiding this comment

SuZhou-Joe Mar 28, 2024

Choose a reason for hiding this comment

SuZhou-Joe commented Mar 28, 2024

sejli commented Mar 28, 2024 • edited Loading

SuZhou-Joe commented Mar 28, 2024

sejli commented Mar 28, 2024 • edited Loading

sejli commented Mar 28, 2024 • edited Loading

SuZhou-Joe commented Mar 29, 2024

sejli commented Mar 29, 2024 • edited Loading

sejli commented Mar 28, 2024 •

edited

Loading

sejli commented Mar 28, 2024 •

edited

Loading

sejli commented Mar 28, 2024 •

edited

Loading

sejli commented Mar 29, 2024 •

edited

Loading