Skip to content

Commit b56fb41

Browse files
mingshlmingshl
committed
add new use cases
Signed-off-by: Mingshi Liu <mingshl@amazon.com>
1 parent fb8d9d3 commit b56fb41

File tree

1 file changed

+371
-1
lines changed

1 file changed

+371
-1
lines changed

‎_search-plugins/search-pipelines/ml-inference-search-response.md‎

Lines changed: 371 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -388,4 +388,374 @@ The response confirms that the processor has generated text embeddings in the `p
388388
]
389389
}
390390
}
391-
```
391+
```
392+
393+
### Example: GENAI Use Case
394+
395+
The following example shows you how to configure an `ml_inference` search response processor with a genai model and mapping the model response to the search extension.
396+
397+
Step 0: Host a model
398+
The pre-requisite is a registered genai model in opensearch. For more information about externally hosted models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index/). Here is a sample predict response using a registered model, which requires a prompt and a context field.
399+
400+
```json
401+
POST /_plugins/_ml/models/EOF6wJIBtDGAJRTD4kNg/_predict
402+
{
403+
"parameters": {
404+
"prompt":"\n\nHuman: You are a professional data analysist. You will always answer question based on the given context first. If the answer is not directly shown in the context, you will analyze the data and find the answer. If you don't know the answer, just say I don't know. Context: ${parameters.context.toString()}. \n\n Human: please summarize the documents \n\n Assistant:",
405+
"context":"Dr. Eric Goldberg is a fantastic doctor who has correctly diagnosed every issue that my wife and I have had. Unlike many of my past doctors, Dr. Goldberg is very accessible and we have been able to schedule appointments with him and his staff very quickly. We are happy to have him in the neighborhood and look forward to being his patients for many years to come."
406+
}
407+
}
408+
```
409+
410+
411+
Step 1: Create a pipeline
412+
413+
The following example shows you how to create a search pipeline for a generative AI model. The model requires a context field as input and generates a response. It summarizes the text in the review field and stores the summary in the ext.ml_inference.llm_response field of the search response.
414+
415+
```json
416+
417+
PUT /_search/pipeline/my_pipeline_request_review_llm
418+
{
419+
"response_processors": [
420+
{
421+
"ml_inference": {
422+
"tag": "ml_inference",
423+
"description": "This processor is going to run llm",
424+
"model_id": "EOF6wJIBtDGAJRTD4kNg",
425+
"function_name": "REMOTE",
426+
"input_map": [
427+
{
428+
"context": "review"
429+
}
430+
],
431+
"output_map": [
432+
{
433+
"ext.ml_inference.llm_response": "response"
434+
}
435+
],
436+
"model_config": {
437+
"prompt": "\n\nHuman: You are a professional data analyst. You will always answer questions based on the given context first. If the answer is not directly shown in the context, you will analyze the data and find the answer. If you don't know the answer, just say I don't know. Context: ${parameters.context.toString()}. \n\n Human: please summarize the documents \n\n Assistant:"
438+
},
439+
"ignore_missing": false,
440+
"ignore_failure": false
441+
}
442+
}
443+
]
444+
}
445+
446+
```
447+
{% include copy-curl.html %}
448+
449+
In this configuration:
450+
451+
The model_id specifies the ID of the generative AI model.
452+
The function_name is set to "REMOTE", indicating an externally hosted model.
453+
The input_map maps the review field from the document to the context field expected by the model.
454+
The output_map specifies that the model's response should be stored in ext.ml_inference.llm_response in the search response.
455+
The model_config includes a prompt that instructs the model on how to process the input and generate a summary.
456+
457+
Step 2: Index sample documents
458+
459+
Index some sample documents to test the pipeline:
460+
```json
461+
PUT /review_string_index/_doc/1
462+
{
463+
"review": "always my to go place" ,
464+
"label":"5 stars"
465+
}
466+
467+
PUT /review_string_index/_doc/2
468+
{
469+
"review": "happy visit" ,
470+
"label":"5 stars"
471+
}
472+
473+
PUT /review_string_index/_doc/3
474+
{
475+
"review": "sad place" ,
476+
"label":"1 stars"
477+
}
478+
```
479+
{% include copy-curl.html %}
480+
481+
Step 3: Run the pipeline
482+
483+
Execute a search query using the pipeline:
484+
```json
485+
GET /review_string_index/_search?search_pipeline=my_pipeline_request_review_llm
486+
{
487+
"query": {
488+
"match_all": {}
489+
}
490+
}
491+
```
492+
{% include copy-curl.html %}
493+
494+
Step 4: Examine the response
495+
496+
The response will include the original documents and the generated summary in the ext.ml_inference.llm_response field:
497+
498+
```json
499+
{
500+
"took": 1,
501+
"timed_out": false,
502+
"_shards": {
503+
"total": 1,
504+
"successful": 1,
505+
"skipped": 0,
506+
"failed": 0
507+
},
508+
"hits": {
509+
"total": {
510+
"value": 3,
511+
"relation": "eq"
512+
},
513+
"max_score": 1,
514+
"hits": [
515+
{
516+
"_index": "review_string_index",
517+
"_id": "1",
518+
"_score": 1,
519+
"_source": {
520+
"review": "always my to go place",
521+
"label": "5 stars"
522+
}
523+
},
524+
{
525+
"_index": "review_string_index",
526+
"_id": "2",
527+
"_score": 1,
528+
"_source": {
529+
"review": "happy visit",
530+
"label": "5 stars"
531+
}
532+
},
533+
{
534+
"_index": "review_string_index",
535+
"_id": "3",
536+
"_score": 1,
537+
"_source": {
538+
"review": "sad place",
539+
"label": "1 stars"
540+
}
541+
}
542+
]
543+
},
544+
"ext": {
545+
"ml_inference": {
546+
"llm_response": "Based on the context provided, here is a summary:\n\nThe context includes 3 short phrases or documents:\n\n1. \"always my to go place\" - This suggests that whatever is being referred to is always the place that someone goes to. It could imply that it is a favorite or preferred location. \n\n2. \"happy visit\" - This directly states that visiting wherever is being discussed results in happiness. \n\n3. \"sad place\" - In contrast to the previous two documents, this one indicates that the place makes someone sad.\n\nIn summary, two of the documents have positive connotations about visiting a particular place, implying it is enjoyed. The third document provides a negative view, saying the place results in sadness. Without more context around what \"place\" is being referred to, it is difficult to draw definitive conclusions. But overall the data provided presents both positive and negative assessments of the same unnamed location."
547+
}
548+
}
549+
}
550+
```
551+
{% include copy-curl.html %}
552+
This example demonstrates how the ml_inference search response processor can be used with a generative AI model to provide summarization of search results. The summary is included in the ext field of the search response, allowing for easy access to the AI-generated insights alongside the original search results.
553+
554+
555+
### Example: Rerank Use Case
556+
557+
The following example shows you how to configure an `ml_inference` search response processor with a text similarity model.
558+
559+
560+
Step 0: Host a model
561+
The pre-requisite is a registered genai model in opensearch. For more information about externally hosted models, see [Connecting to externally hosted models]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/index/). Here is a sample predict response using a registered model, which requires a prompt and a context field.
562+
563+
```json
564+
POST _plugins/_ml/models/tg5p1ZEB4iWlnHsIh2U9/_predict
565+
{
566+
"query_text": "today is sunny",
567+
"text_docs": [
568+
"how are you",
569+
"today is sunny",
570+
"today is july fifth",
571+
"it is winter"
572+
]
573+
}
574+
```
575+
576+
The model returns similarity scores for each input document:
577+
578+
```json
579+
{
580+
"inference_results": [
581+
{
582+
"output": [
583+
{
584+
"name": "similarity",
585+
"data_type": "FLOAT32",
586+
"shape": [1],
587+
"data": [-11.055183]
588+
}
589+
]
590+
},
591+
{
592+
"output": [
593+
{
594+
"name": "similarity",
595+
"data_type": "FLOAT32",
596+
"shape": [1],
597+
"data": [8.969885]
598+
}
599+
]
600+
},
601+
{
602+
"output": [
603+
{
604+
"name": "similarity",
605+
"data_type": "FLOAT32",
606+
"shape": [1],
607+
"data": [-5.736347]
608+
}
609+
]
610+
},
611+
{
612+
"output": [
613+
{
614+
"name": "similarity",
615+
"data_type": "FLOAT32",
616+
"shape": [1],
617+
"data": [-10.0452175]
618+
}
619+
]
620+
}
621+
]
622+
}
623+
```
624+
{% include copy-curl.html %}
625+
626+
Step 1: Index sample documents
627+
628+
Create an index and add some sample documents:
629+
630+
```json
631+
PUT /demo-index-0/_doc/1
632+
{
633+
"dairy": "how are you"
634+
}
635+
636+
PUT /demo-index-0/_doc/2
637+
{
638+
"dairy": "today is sunny"
639+
}
640+
641+
PUT /demo-index-0/_doc/3
642+
{
643+
"dairy": "today is july fifth"
644+
}
645+
646+
PUT /demo-index-0/_doc/4
647+
{
648+
"dairy": "it is winter"
649+
}
650+
651+
```
652+
{% include copy-curl.html %}
653+
654+
Step 2: Create a search pipeline
655+
656+
Create a search pipeline that uses the text similarity model:
657+
658+
```json
659+
PUT /_search/pipeline/my_pipeline
660+
{
661+
"response_processors": [
662+
{
663+
"ml_inference": {
664+
"tag": "ml_inference",
665+
"description": "This processor runs ml inference during search response",
666+
"model_id": "tg5p1ZEB4iWlnHsIh2U9",
667+
"model_input": "{ \"text_docs\": ${input_map.text_docs}, \"query_text\": \"${input_map.query_text}\" }",
668+
"function_name": "TEXT_SIMILARITY",
669+
"input_map": [
670+
{
671+
"text_docs": "dairy",
672+
"query_text": "$.query.term.dairy.value"
673+
}
674+
],
675+
"output_map": [
676+
{
677+
"rank_score": "$.inference_results[*].output[*].data"
678+
}
679+
],
680+
"full_response_path": false,
681+
"model_config": {},
682+
"ignore_missing": false,
683+
"ignore_failure": false
684+
}
685+
}
686+
]
687+
}
688+
```
689+
{% include copy-curl.html %}
690+
691+
This pipeline configuration:
692+
693+
Uses the text similarity model with ID tg5p1ZEB4iWlnHsIh2U9.
694+
Maps the dairy field from documents to the text_docs input of the model.
695+
Extracts the query text from the search request's term query.
696+
Maps the model's output to a new rank_score field in the search results.
697+
698+
Step 3: Run the pipeline
699+
700+
Now, perform a search using the created pipeline:
701+
702+
```json
703+
GET /demo-index-0/_search?search_pipeline=my_pipeline
704+
{
705+
"query": {
706+
"term": {
707+
"dairy": {
708+
"value": "today"
709+
}
710+
}
711+
}
712+
}
713+
```
714+
{% include copy-curl.html %}
715+
716+
The response includes the original documents along with their calculated rank scores:
717+
718+
```json
719+
{
720+
"took": 400,
721+
"timed_out": false,
722+
"_shards": {
723+
"total": 1,
724+
"successful": 1,
725+
"skipped": 0,
726+
"failed": 0
727+
},
728+
"hits": {
729+
"total": {
730+
"value": 2,
731+
"relation": "eq"
732+
},
733+
"max_score": 0.71566814,
734+
"hits": [
735+
{
736+
"_index": "demo-index-0",
737+
"_id": "2",
738+
"_score": 0.71566814,
739+
"_source": {
740+
"dairy": "today is sunny",
741+
"rank_score": [
742+
3.6144485
743+
]
744+
}
745+
},
746+
{
747+
"_index": "demo-index-0",
748+
"_id": "3",
749+
"_score": 0.6333549,
750+
"_source": {
751+
"dairy": "today is july fifth",
752+
"rank_score": [
753+
3.6144485
754+
]
755+
}
756+
}
757+
]
758+
}
759+
}
760+
```
761+
{% include copy-curl.html %}

0 commit comments

Comments
 (0)