能在ChatterBot中加入注意力机制吗?

[win10ogod]

我不太了解编程不过试着问了chatgpt。
这是chatgpt的代碼纪录:

from chatterbot import ChatBot
from chatterbot.trainers import ListTrainer
from chatterbot.conversation import Statement
import numpy as np

# Define a function to calculate the dot product attention score
def dot_product_attention(query, values):
    # Calculate the dot product between the query and each value
    scores = np.dot(query, values.T)
    # Apply softmax to the scores to get the attention weights
    attention_weights = np.softmax(scores)
    # Calculate the weighted average of the values using the attention weights
    weighted_values = np.dot(attention_weights, values)
    return weighted_values
def multihead_attention(query, keys, values, num_heads):
    # Split the query, keys, and values into multiple heads
    query_heads = np.array_split(query, num_heads, axis=-1)
    key_heads = np.array_split(keys, num_heads, axis=-1)
    value_heads = np.array_split(values, num_heads, axis=-1)

    # Concatenate the heads along the last axis
    query_heads = np.concatenate(query_heads, axis=-1)
    key_heads = np.concatenate(key_heads, axis=-1)
    value_heads = np.concatenate(value_heads, axis=-1)

    # Calculate the dot product between the query and each key head
    scores = np.dot(query_heads, key_heads.T)
    # Scale the scores by the square root of the number of features
    scores = scores / np.sqrt(query.shape[-1])

    # Apply softmax to the scores to get the attention weights
    attention_weights = np.softmax(scores, axis=-1)

    # Calculate the weighted average of the value heads using the attention weights
    weighted_values = np.dot(attention_weights, value_heads)

    # Concatenate the weighted value heads along the last axis
    weighted_values = np.concatenate(np.split(weighted_values, num_heads, axis=-1), axis=-1)

    return weighted_values

# Read the conversation corpus file
with open('db.txt', 'r', encoding='utf-8') as f:
    corpus = f.readlines()

# Create a ChatBot instance and train it
my_bot = ChatBot(input('请输入ChatBot名称：'))
trainer = ListTrainer(my_bot)

print('开始训练！')

# Train the ChatBot instance with each conversation in the corpus
for conversation in corpus:
    # Split the conversation into two statements
    statements = conversation.strip().split('\t')
    if len(statements) == 2:
        # Create a Statement object for each statement
        statement_1 = Statement(text=statements[0])
        statement_2 = Statement(text=statements[1])
        # Train the ChatBot with the two statements as a pair
        trainer.train([statement_1, statement_2])
    else:
        # If the conversation is not in the expected format, skip it
        continue

print('训练完毕！')

[cdfmlr]

我觉得这个想法很有意思。

但 ChatterBot 本是简单的数据库检索、通过相似语句匹配、得到输出答句，这是个完全的浅层学习过程（甚至好像没有用到向量和矩阵）。在这样的系统中引入深度的注意力机制是否值得。

在你编辑前给的第一版代码里，似乎是完全用 PyTorch 重写了 ChatterBot 推理的整个过程。只是把原来的一些通过编程实现的匹配、检索工作换成了基于神经网络的向量、概率实现，我觉得二者也许大概是等价的。

而后面改过的这版代码似乎更没有意义了，手写了一种 attention，然后似乎就没有然后了，没看懂 🙈

[win10ogod]

你好，此项目的ChatterBot如何实现，还有如何使用呢？

[win10ogod]

然后原代碼取自XzaiCloud/AI-Vtuber項目，这是新版的代码
`from chatterbot import ChatBot
from chatterbot.trainers import ListTrainer
from chatterbot.conversation import Statement
import numpy as np

Define a function to calculate the dot product attention score

def dot_product_attention(query, values):
# Calculate the dot product between the query and each value
scores = np.dot(query, values.T)
# Apply softmax to the scores to get the attention weights
attention_weights = np.softmax(scores)
# Calculate the weighted average of the values using the attention weights
weighted_values = np.dot(attention_weights, values)
return weighted_values

Read the conversation corpus file

with open('db.txt', 'r', encoding='utf-8') as f:
corpus = f.readlines()

Create a ChatBot instance and train it

my_bot = ChatBot(input('请输入ChatBot名称：'))
trainer = ListTrainer(my_bot)

print('开始训练！')

Train the ChatBot instance with each conversation in the corpus

for conversation in corpus:
# Split the conversation into two statements
statements = conversation.strip().split('\t')
if len(statements) == 2:
# Create a Statement object for each statement
statement_1 = Statement(text=statements[0])
statement_2 = Statement(text=statements[1])

    # Convert the text of the statements to numpy arrays
    query = np.array(statement_1.text.split())
    values = np.array(statement_2.text.split())

    # Calculate the attention weighted input vector using dot_product_attention
    input_vector = dot_product_attention(query, values)

    # Train the ChatBot with the attention weighted input vector
    trainer.train([Statement(text=input_vector)])
else:
    # If the conversation is not in the expected format, skip it
    continue

print('训练完毕！')`
這個程式碼示例將每對對話語句作為輸入向量傳遞給 dot_product_attention 函數，該函數計算注意力加權的輸入向量。然後，注意力加權的輸入向量被轉換成 Statement 物件，並傳遞給 train() 方法，以便用於訓練聊天機器人。

[cdfmlr]

你好，此项目的ChatterBot如何实现，还有如何使用呢？

这个项目目前主要使用 OpenAI 的 ChatGPT API，ChatterBot 只是作为备用机制。使用 ChatterBot 的实现在: https://github.com/cdfmlr/musharing_chatbot/ ，只是对 ChatterBot 项目（ https://github.com/RaSan147/ChatterBot_update ）的 API 封装。

然后原代碼取自XzaiCloud/AI-Vtuber項目，这是新版的代码

@XzaiCloud 你在 ChatterBot 加了注意力机制？😱

[0x648]

加了可能忘了
我最近忙死了，啥都记不到

[cdfmlr]

啊这

然后原代碼取自XzaiCloud/AI-Vtuber項目，这是新版的代码

@win10ogod XzaiCloud/AI-Vtuber 似乎没有这个源码？😭

[0x648]

我也记得没加

[0x648]

我更推荐使用blenderbot

[0x648]

chatterbot有点过时了

[win10ogod]

blenderbot 有中文的项目吗?
一直想要找中文的blenderbot