bug: The ai-prompt-decorator plugin automatically appends the user's input history.

[hw872715125]

Current Behavior

I have deployed APISIX using Docker, and I am using ai-proxy and ai-prompt-decorator. During my testing of the API, I found that when ai-prompt-decorator is used, the user's historical input information is automatically appended to the messages in the logs.

When I do not use the ai-prompt-decorator plugin, the user's historical input content is not automatically appended.

Expected Behavior

Normally, the ai-prompt-decorator plugin should only insert the system prompt, and should not append the user's historical input content.

Error Logs

No response

Steps to Reproduce

This is my route configuration.

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${admin_key}" \
  -d '{
    "id": "ai-chat-route",
    "uri": "/v1/chat/completions/daily",
    "hosts": ["test.webber.com"],
    "methods": ["POST", "OPTIONS"],
    "plugins": {
        "limit-count": {
            "count": 3,
            "time_window": 5,
            "key_type": "var_combination",
            "key": "$remote_addr $http_user_id",
            "rejected_code": 429,
            "rejected_msg": "Too many requests",
            "policy": "local"
          },
          "cors": {},
          "jwt-auth": {
            "header": "Authorization",
            "hide_credentials": true
          },
          "prometheus": {
            "prefer_name": true
          },
          "file-logger": {
            "path": "logs/file.log",
            "include_req_body": true,
            "include_resp_body": true
           },
          "ai-proxy": {
            "provider": "openai-compatible",
            "auth": {
              "header": {
                "Authorization": "Bearer '"${api_key}"'"
              }
            },
            "override": {
              "endpoint": "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"
            },
            "logging": {
                "summaries": true,
                "payloads": true
            },
            "timeout": 180000
          },
          "ai-prompt-decorator": {
            "prepend":[
              {
                "role": "system",
                "content": "'"${fallen_angle_daily_prompt}"'"
              }
            ]
          }
      }
  }'

This is my API test code.

var myHeaders = new Headers();

myHeaders.append("Content-Type", "application/json");
myHeaders.append("Authorization", "jjww")

var raw = JSON.stringify({
   "model": "qwen-max-latest",
   "messages": [
      {
         "role": "user",
         "content": "bye"
      }
   ],
   "temperature": 0.7,
   "top_p": 0.8,
   "enable_search": true,
   "search_options": {
      "forced_search": true
   },
   "stream": true,
   "stream_options": {
      "include_usage": true
   }
});

var requestOptions = {
   method: 'POST',
   headers: myHeaders,
   body: raw,
   redirect: 'follow'
};

fetch("https://test.webber.com/v1/chat/completions/daily", requestOptions)
   .then(response => response.text())
   .then(result => console.log(result))
   .catch(error => console.log('error', error));

Environment

[hw872715125]

help～Is there any solution?

[hw872715125]

help! @dosu

[dosubot]

The ai-prompt-decorator plugin is designed to prepend or append configured prompts to the messages array, but it does not automatically add user input history unless that history is already present in the incoming payload or explicitly configured. The plugin's code and documentation confirm that it simply inserts your configured system prompt before the existing messages and preserves the original user messages as they are, without merging or generating history on its own (source, docs).

If you see historical user input being appended, it's likely coming from the payload sent to APISIX or from another plugin (like ai-proxy) that might be merging history. To isolate the issue, try sending a minimal payload and enabling only the ai-prompt-decorator plugin, then compare the request and resulting messages. If you want help analyzing a specific payload or configuration, feel free to share more details!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[hw872715125]

The ai-prompt-decorator plugin is designed to prepend or append configured prompts to the messages array, but it does not automatically add user input history unless that history is already present in the incoming payload or explicitly configured. The plugin's code and documentation confirm that it simply inserts your configured system prompt before the existing messages and preserves the original user messages as they are, without merging or generating history on its own (source, docs).

If you see historical user input being appended, it's likely coming from the payload sent to APISIX or from another plugin (like ai-proxy) that might be merging history. To isolate the issue, try sending a minimal payload and enabling only the ai-prompt-decorator plugin, then compare the request and resulting messages. If you want help analyzing a specific payload or configuration, feel free to share more details!

To reply, just mention @dosu.

How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other    

Please provide an example? For instance, the demo route.@dosu

[hw872715125]

@dosu Can the ai-proxy plugin not be used together with the ai-prompt-decorator plugin?

[dosubot]

Yes, you can use the ai-proxy and ai-prompt-decorator plugins together in APISIX. The official documentation provides example configurations where both plugins are enabled on the same route, with ai-proxy handling the AI provider connection and ai-prompt-decorator managing prompt injection. There are no documented compatibility issues or limitations when using them together—they are designed to be composable and work in sequence as configured on the route. For reference, see the example in the docs where both plugins are used together on a single route here.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[hw872715125]

This is my test route configuration.

curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
  -H "X-API-KEY: ${admin_key}" \
  -d '{
    "id": "ai-chat-test",
    "uri": "/v1/chat/completions/t",
    "hosts": ["test.webber.com"],
    "methods": ["POST"],
    "plugins": {
      "ai-proxy": {
        "provider": "openai-compatible",
        "auth": {
          "header": {
            "Authorization": "Bearer '"${api_key}"'"
          }
        },
        "override": {
          "endpoint": "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions"
        },
        "timeout": 180000
      },
      "ai-prompt-decorator": {
        "prepend":[
          {
            "role": "system",
            "content": "task"
          }
        ]
      }
    }
  }'

I sent the following messages one by one in the messages array: "How are you", "What are you doing now?", and "How many questions did I ask you in total, and what were they exactly?" — please note that these were not sent all at once in a single message. Please check my last test for details.

I called the API using Apifox, and the answer provided by the AI clearly indicates that, without including the user's historical messages in the `messages` array, the AI would have no way of knowing what I had asked before. However, I did not include any historical conversation messages when making the request in Apifox. @dosu

[dosubot]

Based on the official documentation and plugin source code, neither the ai-proxy nor the ai-prompt-decorator plugins in APISIX will automatically append user history to the messages array or persist chat context across requests—they only modify the payload as explicitly configured (for example, by prepending a system prompt) and forward it to the upstream AI provider as received reference reference.

If the AI is recalling previous questions without them being present in the current messages array, this is not expected behavior from APISIX or these plugins. Possible causes include: (1) the client (such as Apifox) is automatically sending previous messages, (2) the upstream AI provider is maintaining session state, or (3) another plugin or middleware is modifying the payload.

To troubleshoot, I recommend:

1. Enable detailed request/response logging in APISIX (e.g., with file-logger and include_req_body/include_resp_body) to inspect the exact payload sent upstream.
2. Try sending a single, stateless request using a minimal client like curl or Postman to rule out client-side session features.
3. Double-check that no other plugins or custom logic are active on your route or globally.

If you share the logged upstream payload for a single request, I can help analyze whether history is being injected before it leaves APISIX.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[hanqingwu]

I think ai-prompt-decorator.lua should not append history .

local function decorate(conf, body_tab)
    local new_messages = conf.prepend or EMPTY
    for _, message in pairs(body_tab.messages) do
        core.table.insert_tail(new_messages, message)
    end

    for _, message in pairs(conf.append or EMPTY) do
        core.table.insert_tail(new_messages, message)
    end

    body_tab.messages = new_messages
end

function _M.insert_tail(tab, ...)
    local idx = #tab
    for i = 1, select('#', ...) do
        idx = idx + 1
        tab[idx] = select(i, ...)
    end

    return idx
end

[hw872715125]

I think ai-prompt-decorator.lua should not append history .

local function decorate(conf, body_tab)
    local new_messages = conf.prepend or EMPTY
    for _, message in pairs(body_tab.messages) do
        core.table.insert_tail(new_messages, message)
    end

    for _, message in pairs(conf.append or EMPTY) do
        core.table.insert_tail(new_messages, message)
    end

    body_tab.messages = new_messages
end

function _M.insert_tail(tab, ...)
    local idx = #tab
    for i = 1, select('#', ...) do
        idx = idx + 1
        tab[idx] = select(i, ...)
    end

    return idx
end

我不太懂lua，请问要怎么修复呢@hanqingwu

[hw872715125]

It is indeed an issue with the function decorate, which keeps appending user input infinitely. I have created a custom plugin similar to ai-prompt-decorator and replaced the original ai-prompt-decorator with it.
thank you! @hanqingwu

[SkyeYoung]

@hw872715125 I couldn't reproduce the situation you described. Please take a look at the test process and configuration below to see if there are any issues, or provide a more accurate process so that I can assist in resolving the problem.

Please read the next comment

I have tested in both 3.12.0 and 3.13.0, and the configuration is as follows:

#apisix.yaml
routes:
  - id: ai-chat-test
    uri: /v1/chat/completions/t
    methods:
      - POST
    plugins:
      ai-proxy:
        provider: openai-compatible
        auth:
          header:
            Authorization: "Bearer token"
        override:
          endpoint: "https://openrouter.ai/api/v1/chat/completions"
        timeout: 180000
        logging:
          summaries: true
          payloads: true
      ai-prompt-decorator:
        prepend:
          - role: system
            content: task
      file-logger:
        path: "/usr/local/apisix/logs/file.log"
        include_req_body: true
        include_resp_body: true
#END

#config.yaml
apisix:
  node_listen: 9080
  enable_admin: true
  enable_admin_cors: true

deployment:
  role: data_plane
  role_data_plane:
    config_provider: yaml
  admin:
    allow_admin:
      - 0.0.0.0/0
    admin_key:
      - name: "admin"
        key: edd1c9f034335f136f87ad84b625c8f1
        role: admin
    admin_listen:
      ip: 0.0.0.0
      port: 9180

plugins:
  - ai-proxy
  - ai-prompt-decorator
  - file-logger

# docker-compose.yaml
services:
  apisix:
    image: apache/apisix:3.12.0-debian
    container_name: apisix
    restart: always
    volumes:
      - ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml:ro
      - ./apisix_conf/apisix.yaml:/usr/local/apisix/conf/apisix.yaml:ro
      # logs
      - ./logs/error.log:/usr/local/apisix/logs/error.log
      - ./logs/access.log:/usr/local/apisix/logs/access.log
      - ./logs/file.log:/usr/local/apisix/logs/file.log
    ports:
      - "9080:9080"
      - "9180:9180"

Made two requests:

$ curl -X POST "http://localhost:9080/v1/chat/completions/t"   -H "Content-Type: application/json"   -d '{
    "model": "openai/gpt-oss-20b:free",
    "messages": [
      {
        "role": "user",
        "content": "test111"
      }
    ]
  }'


$ curl -X POST "http://localhost:9080/v1/chat/completions/t"   -H "Content-Type: application/json"   -d '{
    "model": "openai/gpt-oss-20b:free",
    "messages": [
      {
        "role": "user",
        "content": "test222"
      }
    ]
  }'

However, the situation described in your first picture was not found in file.log: