[BUG] Memory - Inputs validation needed

[Zhangxunmt]

What is the bug?

1. When creating a conversation with {"name":null} in the request body, the response is 500 with error

"type":"illegal_state_exception","reason":"Can't get text on a VALUE_NULL at 2:8".

But this should be an un expected input from user and throw 4xx exception.

2. When fuzzing the JSON body of the CreateConversation endpoint, if we give a large number value as parameter of the request instead of using a JSON body with the name parameter, the response is

""root_cause":[{"type":"stream_constraints_exception","reason":"Number length (5628) exceeds the maximum length (1000)"}]".

But a 4xx status code should be used in the response because it was the user who sent a non-valid request with a large number that exceeded the maximum length permitted.

3. Non well-deleted memory - Deleted conversation creates a null pointer exception when modified or after a modification is made to it or its interactions.

POST /_plugins/_ml/memory
{
    "name": "Create a new Memory - VAPT - frpalma"
}
#Response
{
  "memory_id": "cNyaZ40BZFGCrcYyQBQ2"
}
# Delete the memory
DELETE /_plugins/_ml/memory/cNyaZ40BZFGCrcYyQBQ2
# Response
{
  "success": true
}
# Take memory back to "live"
PUT /_plugins/_ml/memory/cNyaZ40BZFGCrcYyQBQ2
{
"name": "Take memory from 'graveyard'"
}
# Response
{
  "_index": ".plugins-ml-memory-meta",
  "_id": "cNyaZ40BZFGCrcYyQBQ2",
  "_version": 23,
  "result": "updated",
  "forced_refresh": true,
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "_seq_no": 273,
  "_primary_term": 1
}

# Request
GET /_plugins/_ml/memory/cNyaZ40BZFGCrcYyQBQ2/messages

# Response
{
  "error": {
    "root_cause": [
      {
        "type": "null_pointer_exception",
        "reason": "text"
      }
    ],
    "type": "null_pointer_exception",
    "reason": "text"
  },
  "status": 500
}

4. Process Response - Empty fields on interactions creates a null pointer exception

# Create a new memory
POST /_plugins/_ml/memory
{
    "name": "Create a new Memory - VAPT - frpalma"
}
# Response
{
  "memory_id": "99t9YY0BZFGCrcYyWOL9"
}
# Create an empty interaction in the memory
POST /_plugins/_ml/memory/99t9YY0BZFGCrcYyWOL9/messages
{
}
# Response
{
  "message_id": "_dt9YY0BZFGCrcYyieLd"
}
# Observe that message allows null data in the whole interaction
GET /_plugins/_ml/memory/99t9YY0BZFGCrcYyWOL9/messages
# Response
{
  "interactions": [
    {
      "memory_id": "99t9YY0BZFGCrcYyWOL9",
      "message_id": "_dt9YY0BZFGCrcYyieLd",
      "create_time": "2024-01-31T21:47:13.754888289Z",
      "input": null,
      "prompt_template": null,
      "response": null,
      "origin": null,
      "additional_info": {}
    }
  ]
}

# Creating a RAG pipeline in an index and using an empty conversation yields a 5XX error
GET /my_rag_test_data/_search?search_pipeline=RagPipelineVAPT
{
    "query" : {"match_all": {}},
    "ext": {
        "generative_qa_parameters": {
            "llm_model": "gpt-3.5-turbo",
            "llm_question":"Tell me who did the first question",
            "conversation_id": "99t9YY0BZFGCrcYyWOL9",
            "context_size": 5,
            "interaction_size": 5,
            "timeout": 100000
        }
    }
}
# Response
{
  "error": {
    "root_cause": [
      {
        "type": "null_pointer_exception",
        "reason": null
      }
    ],
    "type": "null_pointer_exception",
    "reason": null
  },
  "status": 500
}

5. Multiple null exception when arguments are missing through calling RAG pipeline

# llm_question missing
GET /my_rag_test_data/_search?search_pipeline=RagPipelineVAPT
{
    "query" : {"match_all": {}},
    "ext": {
        "generative_qa_parameters": {
            "llm_question":"Tell me who did the first question",
            "conversation_id": "99t9YY0BZFGCrcYyWOL9",
            "context_size": 5,
            "interaction_size": 5,
            "timeout": 100000
        }
    }
}
# Response
{
  "error": {
    "root_cause": [
      {
        "type": "null_pointer_exception",
        "reason": "llm_question must not be null."
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "my_rag_test_data",
        "node": "3LQVWg93TseOKLvxqd7fMw",
        "reason": {
          "type": "null_pointer_exception",
          "reason": "llm_question must not be null."
        }
      }
    ],
    "caused_by": {
      "type": "null_pointer_exception",
      "reason": "llm_question must not be null.",
      "caused_by": {
        "type": "null_pointer_exception",
        "reason": "llm_question must not be null."
      }
    }
  },
  "status": 500
}
# context_size / interaction_size / timeout missing
GET /my_rag_test_data/_search?search_pipeline=RagPipelineVAPT
{
    "query" : {"match_all": {}},
    "ext": {
        "generative_qa_parameters": {
            "llm_model": "gpt-3.5-turbo",
            "llm_question":"Tell me who did the first question",
            "conversation_id": "99t9YY0BZFGCrcYyWOL9",
            "interaction_size": 5,
            "timeout": 100000
        }
    }
}
# Response / error varies in the argument missing, but still a intValue call receiving null
{
  "error": {
    "root_cause": [
      {
        "type": "null_pointer_exception",
        "reason": "Cannot invoke \"java.lang.Integer.intValue()\" because \"this.contextSize\" is null"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "my_rag_test_data",
        "node": "3LQVWg93TseOKLvxqd7fMw",
        "reason": {
          "type": "null_pointer_exception",
          "reason": "Cannot invoke \"java.lang.Integer.intValue()\" because \"this.contextSize\" is null"
        }
      }
    ],
    "caused_by": {
      "type": "null_pointer_exception",
      "reason": "Cannot invoke \"java.lang.Integer.intValue()\" because \"this.contextSize\" is null",
      "caused_by": {
        "type": "null_pointer_exception",
        "reason": "Cannot invoke \"java.lang.Integer.intValue()\" because \"this.contextSize\" is null"
      }
    }
  },
  "status": 500
}

It is recommended to implement error handling that does not present internal errors to the customer.

6. GetInteractions - Requesting messages from a conversation that belongs to another user
   When we were testing the authorization of the GetInteractions operation, we used an account of a user to request the messages of a conversation (memory) that belongs to another user:

{"error":{"root_cause":[{"type":"security_exception","reason":"User [user1] does not have access to conversation utv1UY0BZFGCrcYygE1R"}],"type":"security_exception","reason":"User [user1] does not have access to conversation utv1UY0BZFGCrcYygE1R"},"status":500}

5xx codes are used to indicate server errors and, in this case, it is the user who requested a resource that does not have access to. Therefore a 4xx status code should be used in the response

7. CreateInteraction - Creating messages for conversations that belongs to another users
   When we were testing the authorization of the CreateInteraction operation, we used an account of a user in an attempt to create a message in a conversation (memory) that belongs to another user:

{"error":{"root_cause":[{"type":"security_exception","reason":"User [user1] does not have access to conversation utv1UY0BZFGCrcYygE1R"}],"type":"security_exception","reason":"User [user1] does not have access to conversation utv1UY0BZFGCrcYygE1R"},"status":500}

5xx codes are used to indicate server errors and, in this case, it is the user who requested a resource that does not have access to. Therefore a 4xx status code should be used in the response

8. GetConversation - Requesting a conversation (memory) that belongs to a different user
   When we were testing the authorization of the GetConversation operation, we used an account of a user to request a conversation (memory) that belongs to another user:

{"error":{"root_cause":[{"type":"security_exception","reason":"User [user1] does not have access to conversation utv1UY0BZFGCrcYygE1R"}],"type":"security_exception","reason":"User [user1] does not have access to conversation utv1UY0BZFGCrcYygE1R"},"status":500}

5xx codes are used to indicate server errors and, in this case, it is the user who request a resource that does not have access to. Therefore a 4xx status code should be used in the response

[dhrubo-os]

Closing the issue in favor of #2034

[Zhangxunmt]

It’s not fully complete yet. Only the first item is done unfortunately.