Long-term memory via vector database

[engma-linguistics]

https://www.mlq.ai/gpt-4-pinecone-website-ai-assistant/

Using a vector database like Pinecone, a user ought be able to store memory for the 'bot and query it as needed.

[jantic]

GREAT idea. Reason why this is great to clarify for others: It'll make much more efficient use of limited context window when calling GPT to restrict what's loaded into there from long term memory to be stuff that is likely most relevant. And it'll allow for pretty much indefinite length of long term memory that can be used. Think of this as an "associative memory", where the association comes from figuring out what in long term memory is closest in conceptual space compared to the current context.

[ryanpeach]

Seconded. I think this is a must.

[dschonholtz]

I have a PR for this see #122

[ResourceHog]

Found this open source vector db https://github.com/spotify/annoy , thoughts?

[ResourceHog]

also this, https://github.com/milvus-io/milvus

[StableInquest]

#769

[chyezh]

also this, https://github.com/milvus-io/milvus

i have a PR for this, #801

[richbeales]

implemented