[Search] Inference Management FTR Tests #188438

Samiul-TheSoccerFan · 2024-07-16T13:24:48Z

Summary

FTR tests for Inference Management UI. Due to a race condition between license subscription and app registration, we are avoiding these tests for Stack only.

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios
Flaky Test Runner was used on any tests changed

…fix'

TattdCodeMonkey

I still need to pull this down and test them, but if we're enabling the feature for ES3 then the tests should not be disabled in MKI IMO

TattdCodeMonkey · 2024-07-16T14:22:44Z

x-pack/test_serverless/functional/test_suites/search/inference_management.ts

+  const ml = getService('ml');
+
+  describe('Serverless Inference Management UI', function () {
+    this.tags('skipMKI');


if we're enabling the feature flag then we want to remove this tag. Enabling the feature means we WANT to run these against MKI.

We should verify the tests against MKI before merging this.

TattdCodeMonkey · 2024-07-16T14:25:23Z

config/serverless.es.yml

@@ -17,17 +17,17 @@ xpack.osquery.enabled: false
 ## Fine-tune the search solution feature privileges. Also, refer to `serverless.yml` for the project-agnostic overrides.
 xpack.features.overrides:
  ### Dashboards feature is moved from Analytics category to the Search one.
-  dashboard.category: "enterpriseSearch"
+  dashboard.category: 'enterpriseSearch'


do we need these quote changes?

We do not need these quote changes. These go added when I rand lint command so I assumed something changed with the eslint configuration which forced the file to update.

kibanamachine · 2024-07-16T18:18:47Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#6558

[✅] x-pack/test_serverless/functional/test_suites/search/config.ts: 25/25 tests passed.

see run history

Samiul-TheSoccerFan · 2024-07-16T19:24:49Z

@elasticmachine merge upstream

elasticmachine · 2024-07-16T20:15:56Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 63c3b2a
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-188438-63c3b2a38df6

Failed CI Steps

FTR Configs #66

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`searchInferenceEndpoints`	84.7KB	85.3KB	+591.0B

History

💛 Build #221857 was flaky b84aa85
💔 Build #221761 failed ed476b7

Samiul-TheSoccerFan · 2024-07-17T18:20:53Z

Tests ran again MKI

Samiul-TheSoccerFan · 2024-07-17T18:21:43Z

@TattdCodeMonkey : can you please give a review again?

stale

saikatsarkar056 · 2024-07-18T18:06:44Z

config/serverless.es.yml

@@ -65,7 +65,7 @@ data_visualizer.resultLinks.fileBeat.enabled: false
 xpack.searchPlayground.ui.enabled: true

 # Search InferenceEndpoints
-xpack.searchInferenceEndpoints.ui.enabled: false
+xpack.searchInferenceEndpoints.ui.enabled: true


Based on the recommendation #5, we may need to hide the Inference Endpoints page in the servereless environment.

yes, we will not be merging this PR as the requirements changed

@Samiul-TheSoccerFan we can merge this PR, we just need to not enable the FF. instead run the new FTRs in the feature flag config and only enable it there.

saikatsarkar056 · 2024-07-18T18:11:38Z

x-pack/test/functional/apps/search_inference_management/inference_management.ess.ts

+            num_threads: 1,
+          },
+        };
+        await ml.api.createInferenceEndpoint('endpoint-1', 'sparse_embedding', modelConfig);


This endpoint was created in the ‘before’ block. We may need to write some cleanup code in the ‘after’ block to delete the endpoint and remove the underlying trained model if it was added during the creation of the endpoint.

I’m suggesting this because we previously had a CI/CD failure due to not cleaning up created resources. See this PR as an example of cleanup.

At the last test, we removed the inference endpoint from the UI. Do we still need to do the cleanup or it will be handled through the application itself?

Please check whether creation of inference endpoint adds any new trained model. If so, we need to clean up the trained model.

saikatsarkar056 · 2024-07-18T18:12:39Z

x-pack/test_serverless/functional/test_suites/search/inference_management.ts

+            num_threads: 1,
+          },
+        };
+        await ml.api.createInferenceEndpoint('endpoint-1', 'sparse_embedding', modelConfig);


Similar to this comment, we need to write some clean-up in the 'after' block.

Samiul-TheSoccerFan · 2024-07-22T14:12:09Z

deprioritizing this PR as the plugin flow is expected to be changed. We will revisit this PR once we decide to move forward in Serverless.

jeramysoucy · 2024-07-23T19:41:10Z

@Samiul-TheSoccerFan re:

deprioritizing this PR as the plugin flow is expected to be changed. We will revisit this PR once we decide to move forward in Serverless.

Could you switch this back to a draft PR?

Samiul-TheSoccerFan added 9 commits July 15, 2024 09:53

Adding ftr test for inference navigation

bcaf6ac

Adding test ids for inference tabular view

7a5f4ad

Adding test ids for empty inference page

ce80b32

Adding inference management ftr tests for ES3

66e5608

Adding inference tests for ESS

9dfbc15

Refactoring to share page objects between ES3 and ESS

540997f

Adding link to load inference in ESS

7df0806

skipping inference ftr tests due to race condition

0c39594

Adding comment on why the tests are skipping

e27012e

Samiul-TheSoccerFan requested review from a team as code owners July 16, 2024 13:24

Samiul-TheSoccerFan added release_note:skip Skip the PR/issue when compiling release notes Team:Search v8.15.0 v8.16.0 and removed v8.15.0 labels Jul 16, 2024

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

ed476b7

…fix'

jbudz approved these changes Jul 16, 2024

View reviewed changes

TattdCodeMonkey previously requested changes Jul 16, 2024

View reviewed changes

Samiul-TheSoccerFan added 2 commits July 16, 2024 13:18

fix types for ci

8af9b75

Removing MKI tags for ES3

b84aa85

Samiul-TheSoccerFan added the ci:build-serverless-image label Jul 16, 2024

Samiul-TheSoccerFan added the backport:skip This commit does not require backporting label Jul 16, 2024

Merge branch 'main' into inference-ftr-tests

63c3b2a

Samiul-TheSoccerFan requested a review from TattdCodeMonkey July 17, 2024 18:20

jeramysoucy self-requested a review July 17, 2024 18:50

saikatsarkar056 reviewed Jul 18, 2024

View reviewed changes

jeramysoucy removed their request for review July 23, 2024 19:41

Samiul-TheSoccerFan marked this pull request as draft July 23, 2024 19:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Search] Inference Management FTR Tests #188438

[Search] Inference Management FTR Tests #188438

Samiul-TheSoccerFan commented Jul 16, 2024 •

edited

Loading

TattdCodeMonkey left a comment

TattdCodeMonkey Jul 16, 2024

TattdCodeMonkey Jul 16, 2024

Samiul-TheSoccerFan Jul 16, 2024

kibanamachine commented Jul 16, 2024

Samiul-TheSoccerFan commented Jul 16, 2024

elasticmachine commented Jul 16, 2024 •

edited

Loading

Samiul-TheSoccerFan commented Jul 17, 2024

Samiul-TheSoccerFan commented Jul 17, 2024

saikatsarkar056 Jul 18, 2024 •

edited

Loading

Samiul-TheSoccerFan Jul 18, 2024

TattdCodeMonkey Jul 19, 2024

saikatsarkar056 Jul 18, 2024 •

edited

Loading

Samiul-TheSoccerFan Jul 18, 2024

saikatsarkar056 Jul 19, 2024

saikatsarkar056 Jul 18, 2024 •

edited

Loading

Samiul-TheSoccerFan commented Jul 22, 2024

jeramysoucy commented Jul 23, 2024

[Search] Inference Management FTR Tests #188438

Are you sure you want to change the base?

[Search] Inference Management FTR Tests #188438

Conversation

Samiul-TheSoccerFan commented Jul 16, 2024 • edited Loading

Summary

Checklist

TattdCodeMonkey left a comment

Choose a reason for hiding this comment

TattdCodeMonkey Jul 16, 2024

Choose a reason for hiding this comment

TattdCodeMonkey Jul 16, 2024

Choose a reason for hiding this comment

Samiul-TheSoccerFan Jul 16, 2024

Choose a reason for hiding this comment

kibanamachine commented Jul 16, 2024

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#6558

Samiul-TheSoccerFan commented Jul 16, 2024

elasticmachine commented Jul 16, 2024 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Async chunks

History

Samiul-TheSoccerFan commented Jul 17, 2024

Samiul-TheSoccerFan commented Jul 17, 2024

saikatsarkar056 Jul 18, 2024 • edited Loading

Choose a reason for hiding this comment

Samiul-TheSoccerFan Jul 18, 2024

Choose a reason for hiding this comment

TattdCodeMonkey Jul 19, 2024

Choose a reason for hiding this comment

saikatsarkar056 Jul 18, 2024 • edited Loading

Choose a reason for hiding this comment

Samiul-TheSoccerFan Jul 18, 2024

Choose a reason for hiding this comment

saikatsarkar056 Jul 19, 2024

Choose a reason for hiding this comment

saikatsarkar056 Jul 18, 2024 • edited Loading

Choose a reason for hiding this comment

Samiul-TheSoccerFan commented Jul 22, 2024

jeramysoucy commented Jul 23, 2024

Samiul-TheSoccerFan commented Jul 16, 2024 •

edited

Loading

elasticmachine commented Jul 16, 2024 •

edited

Loading

saikatsarkar056 Jul 18, 2024 •

edited

Loading

saikatsarkar056 Jul 18, 2024 •

edited

Loading

saikatsarkar056 Jul 18, 2024 •

edited

Loading