[Tests] Migrate VisBuilder functional tests into Dashboards repo #6102

manasvinibs · 2024-03-10T05:33:09Z

Description

Migrating VisBuilder functional tests along with utils and fixtures from FT repo to Dashboards repo as a reference tests for other apps to write more tests in the repo.
Adding commands and constants from FT repo.
Moving test library dependency to dev dependency

Issues Resolved

Testing the changes

Run Cypress Tests for ciGroup11 should run all the tests for visBuilders app in the CI.
https://github.com/opensearch-project/OpenSearch-Dashboards/actions/runs/8220159200/job/22478945672?pr=6102

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

codecov · 2024-03-10T05:46:38Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 67.33%. Comparing base (f103b7e) to head (e1e834e).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6102      +/-   ##
==========================================
+ Coverage   67.31%   67.33%   +0.02%     
==========================================
  Files        3348     3348              
  Lines       64966    64966              
  Branches    10465    10465              
==========================================
+ Hits        43729    43743      +14     
+ Misses      18690    18677      -13     
+ Partials     2547     2546       -1

Flag	Coverage Δ
Linux_1	`31.72% <ø> (ø)`
Linux_2	`55.55% <ø> (ø)`
Linux_3	`44.67% <ø> (+<0.01%)`	⬆️
Linux_4	`35.02% <ø> (ø)`
Windows_1	`31.76% <ø> (+0.02%)`	⬆️
Windows_2	`55.52% <ø> (ø)`
Windows_3	`44.68% <ø> (ø)`
Windows_4	`35.02% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ashwin-pc

LGTM. The security overrides need to be validated but that isnt applicable to the local cypress tests so should not block this PR.

ashwin-pc · 2024-03-11T19:11:08Z

cypress/fixtures/example.json

Do we need this?

ashwin-pc · 2024-03-11T19:12:40Z

cypress/support/index.js

+const resizeObserverLoopErrRe = /^[^(ResizeObserver loop limit exceeded)]/;
+Cypress.on('uncaught:exception', (err) => {
+  /* returning false here prevents Cypress from failing the test */
+  if (resizeObserverLoopErrRe.test(err.message)) {
+    return false;
+  }
+});


Why do we need this?

This was migrated as it is from FT Repo. I think without this we are seeing errors on few of Vis builder tests. We might want to dive deep into the tests which are exposing this behavior but for now keeping the parity with FT repo tests I'm just keeping this logic in place.

ResizeObserver loop limit exceeded
When Cypress detects uncaught errors originating from your application it will automatically fail the current test.
This behavior is configurable, and you can choose to turn this off by listening to the uncaught:exception event.
https://on.cypress.io/uncaught-exception-from-application
at (http://localhost:5601/app/vis-builder/#/?_q=(filters:!(),query:(language:kuery,query:''))&_a=(metadata:(editor:(errors:(),state:clean),originatingApp:dashboards),style:(addLegend:!t,addTooltip:!t,legendPosition:right,type:histogram),ui:(),visualization:(activeVisualization:(aggConfigParams:!(),name:histogram),indexPattern:a031e530-5a88-11ed-a595-f5e6ea9b3826,searchField:''))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:'2021-12-31T08:00:00.000Z',to:'2022-10-02T07:00:00.000Z')):0:0)

Think we should catch these errors instead of skipping it like this 😅

Agreed! I think this is more of a Cypress version specific issue than application and might have been introduced in FT repo as a workaround till they upgrade. I see this thread about the same: cypress-io/cypress#8418
May be as part of Cypress version upgrade effort, we can get rid of similar stuff.

ashwin-pc · 2024-03-12T00:55:59Z

cypress/utils/commands.js

+ * This overwrites the default visit command to authenticate before visiting
+ * webpages if SECURITY_ENABLED cypress env var is true
+ */
+Cypress.Commands.overwrite('visit', (orig, url, options) => {


Have we validated these overrides?

I'm still validating/fixing issues for running security enabled tests and its overrides. But as you said should not be a blocker for this PR or to run local CI.

kavilla · 2024-03-19T21:10:10Z

cypress/utils/commands.js

+};
+
+export const CURRENT_TENANT = {
+  defaultTenant: 'private',


make this global please

Signed-off-by: Manasvini B Suryanarayana <manasvis@amazon.com>

kavilla · 2024-04-09T20:54:41Z

cypress/fixtures/opensearch_dashboards/data.json.txt

@@ -0,0 +1,4123 @@
+{


do you think we can compress these large files and update the helper function to modify it? also, not sure if the ndjson vs json.txt was intentionally i think we should standardize our saved objects

kavilla · 2024-04-09T20:56:43Z

cypress/fixtures/opensearch_dashboards/visBuilder/vis-builder.data.txt

can we standardize the structure of this? might need to be compressed too

BionIT · 2024-06-05T03:38:20Z

Hi @manasvinibs , are we targeting this PR for 2.15 or 2.16?

ananzh · 2024-10-30T19:06:51Z

Convert to draft due to no progress.

manasvinibs requested review from ananzh, kavilla, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, kristenTian, zhongnansu, ZilongX, Flyingliuhub, BSFishy, curq, bandinib-amzn, SuZhou-Joe, ruanyl and BionIT as code owners March 10, 2024 05:33

manasvinibs marked this pull request as draft March 10, 2024 05:33

github-actions bot added the distinguished-contributor label Mar 10, 2024

manasvinibs force-pushed the Issue-5892-1 branch from 956523e to 341f24c Compare March 10, 2024 06:34

manasvinibs marked this pull request as ready for review March 10, 2024 06:35

ashwin-pc self-assigned this Mar 11, 2024

ashwin-pc reviewed Mar 12, 2024

View reviewed changes

ashwin-pc assigned manasvinibs and kavilla Mar 12, 2024

manasvinibs added v2.13.0 v2.14.0 and removed v2.13.0 labels Mar 14, 2024

manasvinibs force-pushed the Issue-5892-1 branch from 341f24c to 8d84da3 Compare March 18, 2024 21:50

manasvinibs force-pushed the Issue-5892-1 branch 2 times, most recently from 8f10eb6 to 4df7360 Compare March 19, 2024 18:10

kavilla reviewed Mar 19, 2024

View reviewed changes

Migrate VisBuilder functional tests into Dashboards repo

e1e834e

Signed-off-by: Manasvini B Suryanarayana <manasvis@amazon.com>

manasvinibs force-pushed the Issue-5892-1 branch from 4df7360 to e1e834e Compare March 20, 2024 20:00

manasvinibs requested a review from xinruiba as a code owner March 20, 2024 20:00

kavilla reviewed Apr 9, 2024

View reviewed changes

kavilla removed the v2.14.0 label Apr 23, 2024

ananzh marked this pull request as draft October 30, 2024 19:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Tests] Migrate VisBuilder functional tests into Dashboards repo #6102

[Tests] Migrate VisBuilder functional tests into Dashboards repo #6102

manasvinibs commented Mar 10, 2024 •

edited

Loading

codecov bot commented Mar 10, 2024 •

edited

Loading

ashwin-pc left a comment

ashwin-pc Mar 11, 2024

manasvinibs Mar 18, 2024

ashwin-pc Mar 11, 2024

manasvinibs Mar 18, 2024 •

edited

Loading

ashwin-pc Mar 19, 2024

manasvinibs Mar 21, 2024

ashwin-pc Mar 12, 2024

manasvinibs Mar 18, 2024

kavilla Mar 19, 2024

kavilla Apr 9, 2024

kavilla Apr 9, 2024

BionIT commented Jun 5, 2024

ananzh commented Oct 30, 2024

[Tests] Migrate VisBuilder functional tests into Dashboards repo #6102

Are you sure you want to change the base?

[Tests] Migrate VisBuilder functional tests into Dashboards repo #6102

Conversation

manasvinibs commented Mar 10, 2024 • edited Loading

Description

Issues Resolved

Testing the changes

Check List

codecov bot commented Mar 10, 2024 • edited Loading

Codecov Report

ashwin-pc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

manasvinibs Mar 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BionIT commented Jun 5, 2024

ananzh commented Oct 30, 2024

manasvinibs commented Mar 10, 2024 •

edited

Loading

codecov bot commented Mar 10, 2024 •

edited

Loading

manasvinibs Mar 18, 2024 •

edited

Loading