[Agentic Memory] Allow other LLMs for Fact Extraction

[rithin-pullela-aws]

Description

• Make it flexible to use any LLM models for fact extraction.
• Adds basic ITs for fact extraction in Add Memory API
• Refactor prompts for better readability and improved output format generation

With this PR we expect the users to pass response_filter in parameters of the LLM connectors and thus allowing more flexible LLM options for fact extraction in Agentic Memory. This also means we can have a more flexible connector request body without tying to only claude models.

To use a Claude model:

{
  "name": "Bedrock infer model",
  "function_name": "remote",
  "description": "LLM model for memory processing",
  "connector": {
    "name": "Amazon Bedrock Connector: LLM",
    "description": "The connector to bedrock Claude 3.7 sonnet model",
    "version": 1,
    "protocol": "<AWS_REGION>",
    "parameters": {
      "region": "us-east-1",
      "service_name": "bedrock",
      "max_tokens": 8000,
      "temperature": 1,
      "anthropic_version": "bedrock-2023-05-31",
      "model": "us.anthropic.claude-3-5-sonnet-20240620-v1:0",
      "response_filter": "$.content[0].text"
    },
    "credential": {
        "access_key": <AWS_ACCESS_KEY>,
        "secret_key": <AWS_SECRET_KEY>,
        "session_token": <AWS_SESSION_TOKEN>
    },
    "actions": [{
      "action_type": "predict",
      "method": "POST",
      "headers": {"content-type": "application/json"},
      "url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/model/${parameters.model}/invoke",
      "request_body": "{ \"system\": \"${parameters.system_prompt}\", \"anthropic_version\": \"${parameters.anthropic_version}\", \"max_tokens\": ${parameters.max_tokens}, \"temperature\": ${parameters.temperature}, \"messages\": [{\"role\":\"user\",\"content\":[{ \"type\": \"text\", \"text\":\"${parameters.messages}\"}]}]}"
    }]
  }
}

To use Open AI models:

{
    "name": "My OpenAI model: gpt-4o",
    "function_name": "remote",
    "description": "test model",
    "connector": {
        "name": "My openai connector: gpt-4o mini",
        "description": "The connector to openai chat model",
        "version": 1,
        "protocol": "http",
        "parameters": {
            "model": "gpt-4o-mini",
            "response_filter": "$.choices[0].message.content"
        },
        "credential": {
            "openAI_key": <OPENAI_API_KEY>
        },
        "actions": [
            {
                "action_type": "predict",
                "method": "POST",
                "url": "https://api.openai.com/v1/chat/completions",
                "headers": {
                    "Authorization": "Bearer ${credential.openAI_key}"
                },
                "request_body": "{ \"model\": \"${parameters.model}\", \"messages\": [{\"role\":\"system\",\"content\":\"${parameters.system_prompt}\"},{\"role\":\"user\",\"content\":\"${parameters.messages}\"}] }"
            }
        ]
    }
}

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

• New functionality includes testing.
• New functionality has been documented.
• API changes companion pull request created.
• Commits are signed per the DCO using --signoff.
• Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

❌ Patch coverage is 89.47368% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.90%. Comparing base (281c430) to head (c6cf290).

Files with missing lines	Patch %	Lines
...emorycontainer/memory/MemoryProcessingService.java	89.47%	0 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #4159   +/-   ##
=========================================
  Coverage     81.89%   81.90%           
+ Complexity     8879     8873    -6     
=========================================
  Files           762      762           
  Lines         38200    38154   -46     
  Branches       4267     4256   -11     
=========================================
- Hits          31285    31250   -35     
+ Misses         5084     5078    -6     
+ Partials       1831     1826    -5     

Flag	Coverage Δ
ml-commons	81.90% <89.47%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.