[Obs AI Assistant] Add KB user instructions #187607

sorenlouv · 2024-07-04T21:34:56Z

Related to: https://github.com/elastic/obs-ai-assistant-team/issues/157

This makes user instructions available in the UI which makes it possible for end users to create instructions that are appended to the system prompt. Technically the system supports multiple instructions per user, but for now we limit it to a single instruction per user.

From a technical perspective:
User instructions are stored in the knowledge base. They are included in the system prompt if they fit within the token budget. They are different from the existing "contextual" entries that are only included in the context function response (aka not the system message) and only if they semantically matches the user's prompt.

Screenshots

"Edit system prompt" button

"Edit system prompt" flyout

haiku-prompt.mp4

obltmachine · 2024-07-04T21:35:10Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

x-pack/plugins/observability_solution/observability_ai_assistant/server/service/index.ts

...ution/observability_ai_assistant/server/service/util/get_system_message_from_instructions.ts

sorenlouv · 2024-07-04T22:58:44Z

buildkite test this

...rvability_solution/observability_ai_assistant/server/service/knowledge_base_service/index.ts

…tions # Conflicts: # x-pack/plugins/observability_solution/observability_ai_assistant/server/service/knowledge_base_service/index.ts

sorenlouv · 2024-07-05T21:57:25Z

buildkite test this

…tions

elasticmachine · 2024-07-09T10:06:29Z

💔 Build Failed

Buildkite Build
Commit: 421386f

Failed CI Steps

Quick Checks

History

💔 Build #219948 failed f18dc24
💔 Build #219944 failed f18dc24
💔 Build #219820 failed 4c32316
💔 Build #219734 failed fff5f21478ba611784b4570818d2af80b2587ab7

x-pack/plugins/observability_solution/observability_ai_assistant/server/functions/summarize.ts

...lugins/observability_solution/observability_ai_assistant/server/service/client/index.test.ts

...olution/observability_ai_assistant_management/public/hooks/use_get_knowledge_base_entries.ts

…tions

…ersations

kibana-ci · 2024-08-19T22:51:18Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 465b77b
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-187607-465b77bdd322

Failed CI Steps

FTR Configs #12

Test Failures

[job] [logs] FTR Configs #12 / Visualizations - Group 3 lens app - TSVB Open in Lens Dashboard to TSVB to Lens should convert a by reference TSVB viz to a Lens viz

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`observabilityAiAssistantManagement`	136	139	+3

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`observabilityAIAssistant`	288	292	+4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`observabilityAIAssistantApp`	150.2KB	150.2KB	-18.0B
`observabilityAiAssistantManagement`	87.5KB	91.5KB	+4.0KB
total			+4.0KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`observabilityAIAssistant`	26	27	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`observabilityAIAssistant`	46.8KB	47.0KB	+213.0B
`observabilityAiAssistantManagement`	4.8KB	4.8KB	+48.0B
total			+261.0B

Unknown metric groups

API count

id	before	after	diff
`observabilityAIAssistant`	290	294	+4

References to deprecated APIs

id	before	after	diff
`observabilityAIAssistantApp`	1	0	-1

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

sorenlouv commented Jul 4, 2024

View reviewed changes

x-pack/plugins/observability_solution/observability_ai_assistant/server/service/index.ts Show resolved Hide resolved

sorenlouv commented Jul 4, 2024

View reviewed changes

...ution/observability_ai_assistant/server/service/util/get_system_message_from_instructions.ts Outdated Show resolved Hide resolved

sorenlouv added 2 commits July 5, 2024 00:58

[Obs AI Assistant] Add KB user instructions

f71a8e4

Improve description

f54af02

sorenlouv force-pushed the kb-user-instructions branch from fff5f21 to f54af02 Compare July 4, 2024 22:58

sorenlouv added release_note:enhancement v8.15.0 labels Jul 4, 2024

sorenlouv marked this pull request as ready for review July 5, 2024 07:06

sorenlouv requested a review from a team as a code owner July 5, 2024 07:06

botelastic bot added ci:project-deploy-observability Create an Observability project Team:Obs AI Assistant Observability AI Assistant labels Jul 5, 2024

dgieselaar reviewed Jul 5, 2024

View reviewed changes

...rvability_solution/observability_ai_assistant/server/service/knowledge_base_service/index.ts Outdated Show resolved Hide resolved

sorenlouv added 3 commits July 5, 2024 13:26

Address feedback

7c65ee9

Revert change to requestInstructions

4c32316

Merge branch 'main' of github.com:elastic/kibana into kb-user-instruc…

f18dc24

…tions # Conflicts: # x-pack/plugins/observability_solution/observability_ai_assistant/server/service/knowledge_base_service/index.ts

sorenlouv added 2 commits July 8, 2024 09:49

Merge branch 'main' of github.com:elastic/kibana into kb-user-instruc…

371fb5e

…tions

Add UI for editing system prompt

421386f

sorenlouv requested a review from a team as a code owner July 9, 2024 09:45

pgayvallet approved these changes Jul 10, 2024

View reviewed changes

dgieselaar reviewed Jul 11, 2024

View reviewed changes

x-pack/plugins/observability_solution/observability_ai_assistant/server/functions/summarize.ts Outdated Show resolved Hide resolved

dgieselaar reviewed Jul 15, 2024

View reviewed changes

...lugins/observability_solution/observability_ai_assistant/server/service/client/index.test.ts Outdated Show resolved Hide resolved

dgieselaar reviewed Jul 15, 2024

View reviewed changes

...olution/observability_ai_assistant_management/public/hooks/use_get_knowledge_base_entries.ts Outdated Show resolved Hide resolved

dgieselaar approved these changes Jul 16, 2024

View reviewed changes

sorenlouv added 3 commits August 5, 2024 09:38

Merge branch 'main' into kb-user-instructions

dec0374

Address feedback

063575b

Merge branch 'main' of github.com:elastic/kibana into kb-user-instruc…

14ec494

…tions

Merge branch 'main' of github.com:elastic/kibana into kb-user-instruc…

816e6eb

…tions

sorenlouv added backport v8.15.1 v8.16.0 and removed v8.15.0 v8.15.1 backport labels Aug 14, 2024

sorenlouv added 7 commits August 14, 2024 16:29

Add test that validates user instruction is added to system prompt

c2953b3

Add test to verify other users does not see instruction in their conv…

e433ab7

…ersations

Fik jest test (token count fixes)

3639edf

Undo change to tsconfig.base.json

e238ba0

Use shared isFunctionTitleRequest

61bf0f4

Remove unused import

4f960da

Fix tsc issues

4f0982c

sorenlouv enabled auto-merge (squash) August 15, 2024 14:41

sorenlouv added 11 commits August 15, 2024 22:39

Fix functional test

55a4fd5

Fix functional test

d41849c

Fix jest

20f56fd

Remove context intercept

464b0b8

Merge branch 'main' into kb-user-instructions

d76f1ce

Fix tests by clearing conversations

72f3338

Merge branch 'main' into kb-user-instructions

3004e2d

Merge branch 'main' into kb-user-instructions

ed6ee2c

Merge branch 'main' into kb-user-instructions

ee78461

Cleanup helper functions

39627f1

Change to debug

465b77b

sorenlouv merged commit e9f23aa into elastic:main Aug 19, 2024
23 checks passed

kibanamachine added the backport:skip This commit does not require backporting label Aug 19, 2024

sorenlouv deleted the kb-user-instructions branch August 20, 2024 03:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Obs AI Assistant] Add KB user instructions #187607

[Obs AI Assistant] Add KB user instructions #187607

Uh oh!

sorenlouv commented Jul 4, 2024 •

edited

Loading

Uh oh!

obltmachine commented Jul 4, 2024

Uh oh!

Uh oh!

Uh oh!

sorenlouv commented Jul 4, 2024

Uh oh!

Uh oh!

sorenlouv commented Jul 5, 2024

Uh oh!

elasticmachine commented Jul 9, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kibana-ci commented Aug 19, 2024 •

edited

Loading

API count

References to deprecated APIs

Uh oh!

Uh oh!

Uh oh!

[Obs AI Assistant] Add KB user instructions #187607

[Obs AI Assistant] Add KB user instructions #187607

Uh oh!

Conversation

sorenlouv commented Jul 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Screenshots

Uh oh!

obltmachine commented Jul 4, 2024

🤖 GitHub comments

Uh oh!

Uh oh!

Uh oh!

sorenlouv commented Jul 4, 2024

Uh oh!

Uh oh!

sorenlouv commented Jul 5, 2024

Uh oh!

elasticmachine commented Jul 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Failed CI Steps

History

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kibana-ci commented Aug 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

References to deprecated APIs

History

Uh oh!

Uh oh!

Uh oh!

sorenlouv commented Jul 4, 2024 •

edited

Loading

elasticmachine commented Jul 9, 2024 •

edited

Loading

kibana-ci commented Aug 19, 2024 •

edited

Loading