Implement Local Cache and Redis Memory backend #372

BillSchumacher · 2023-04-07T05:13:47Z

Background

Redis is a free and open source alternative to pinecone.

We had a dicussion here: https://discord.com/channels/1092243196446249134/1093713674679615638

Changes

The memory backend was given an abstract base class.

PineconeMemory now inherits from this ABC.

RedisMemory was added.

Configuration options were added to the Config class.

Test Plan

I had SuperMario in Discord verify I did not break pinecone, I tested redis locally.

Change Safety

I have added tests to cover my changes
I have considered potential risks and mitigations for my changes

I didn't see any existing tests for memory and will attempt to add some at a later point in time.

… of instantiating a new one where used.

BillSchumacher · 2023-04-07T05:25:35Z

I should note that there is no reason why we have to wipe out the memory, unless the intent is to reset state.

We could easily save the memory store's state on exit and load on start, or even per add() call.

BillSchumacher · 2023-04-07T05:49:58Z

In redis this is simple so I went ahead and persisted the vec_num to redis and added an option to wipe on startup as the default behavior, optionally allowing users to keep their state.

aosan · 2023-04-07T06:33:09Z

In redis this is simple so I went ahead and persisted the vec_num to redis and added an option to wipe on startup as the default behavior, optionally allowing users to keep their state.

👍
Allowing for persistence is a great idea for any long running jobs. CloudFlare frequently throws a 401 finishing the session. Warm restarts would need a steady state.

BillSchumacher · 2023-04-07T23:14:15Z

This now implements a local version using numpy.

BillSchumacher · 2023-04-07T23:14:43Z

It is also set to the default, orjson was added as a requirement.

BillSchumacher · 2023-04-07T23:32:33Z

This now avoids import errors for both redis and pinecone. It is possible to remove them from requirements.txt.

dominikdassow · 2023-04-08T00:12:58Z

Hi, I see that both you and @cs0lar in #424 work on a memory backend/factory/interface. Great stuff! I'd suggest having a look into creating a strong combined implementation.

I think @cs0lar's implementation is a bit neater and simpler, and has tests. You, @BillSchumacher, have the two additional providers, Redis and local.

This memory interface is a good step 👍🏻 Happy to help and work together here

supermario-ai · 2023-04-08T00:53:33Z

@BillSchumacher LocalCache is crusing rn. no major errors. ~5 min of run time. Starting Redis smoke test.

supermario-ai · 2023-04-08T01:19:48Z

@BillSchumacher Redis - Pass
@ALL

of all the memory types, in my experience, redis doens't bork segmentation

PC and LocalCache, dont do this well with teh stock entreprenur persona & goals

supermario-ai · 2023-04-08T01:22:44Z

🟢

…ackend

jkoelker

LGTM, I've use this as is for a few test setups without issue. Just a suggestion to default the env vars to make the presumed local case easier to setup.

scripts/config.py

Co-authored-by: Jason Kölker <jason@koelker.net>

BillSchumacher · 2023-04-08T02:04:03Z

Hi, I see that both you and @cs0lar in #424 work on a memory backend/factory/interface. Great stuff! I'd suggest having a look into creating a strong combined implementation.

I think @cs0lar's implementation is a bit neater and simpler, and has tests. You, @BillSchumacher, have the two additional providers, Redis and local.

This memory interface is a good step 👍🏻 Happy to help and work together here

I'm not sure that having a class for a single function makes sense, if other people like that I'm good with whatever.

The tests are a start but we probably need some CI/CD to be able to really test.

Torantulino · 2023-04-08T02:06:38Z

Taking a look at this.
Thank you for your contribution!

ryanpeach · 2023-04-09T01:40:51Z

@Torantulino once you merge this then I'll merge my combo PR back into my qa pr to handle the merge conflict produced, and we will be back in buisness.

scripts/main.py

Remove pinecone config requirement

Torantulino · 2023-04-09T03:30:56Z

Running this causes almost all of AutoGPT outputs to not be properly formatted as JSON for me.

Anyone else experiencing this problem?

BillSchumacher · 2023-04-09T03:51:45Z

Running this causes almost all of AutoGPT outputs to not be properly formatted as JSON for me.

Anyone else experiencing this problem?

I'm not sure the JSON errors are restricted to this branch, the main thing I'm seeing from a local model is just bad output from the model.

relevant memory [[Document {'id': 'bob:0', 'payload': None, 'vector_score': '0.156416118145', 'data': 'Assistant Reply: Understood. My first task is "Search Files" for python scripts. \nResult: Command Error: returned: Unknown command Error: \nHuman Feedback: GENERATE NEXT COMMAND JSON '}]]

Is what's coming back from relevent memory.

Torantulino · 2023-04-09T03:58:53Z

I can see in the readme you mentioned instructions on how to make Redis memory persist between runs, however using your "LocalCache" version, memory is persisting for me by default.

How do users configure this?

Torantulino · 2023-04-09T04:04:39Z

scripts/memory/__init__.py

+            memory = RedisMemory(cfg)
+
+    if memory is None:
+        memory = LocalCache(cfg)


We appear to be missing a call to memory.clear() here.

For the LocalCache

Yeah, there was a lot of people that wanted the memory to persist by default and I just made Pi's implementation work.

I guess a direction one way or the other needs to be picked.

Deleting the memory is easy enough for local, use just delete the json file.

But there's also a memory index if you wanted to start fresh but keep memory from a different session.

Interesting, okay.

Perhaps default persistent memory is a good idea in the near future, however now I think it has the side effect of encouraging the AI to repeat it's errors.

Through my experimenting with this PR, if the AI is reminded of events from it's past were it didn't respond in JSON, it's more likely to do so again.

It might be prudent to not even save those in memory.

BillSchumacher · 2023-04-09T04:11:49Z

Running this causes almost all of AutoGPT outputs to not be properly formatted as JSON for me.

Anyone else experiencing this problem?

    return [result.data for result in results.docs]

Might be better

scripts/memory/__init__.py

BillSchumacher · 2023-04-09T04:29:29Z

Yeah, not saving the bad responses and returning the data seems to be working better for me.

Torantulino · 2023-04-09T04:31:27Z

Agreed, I think that would make more sense to the Agent.
Though benchmarking later on will tell us for sure.

scripts/memory/redismem.py

Torantulino · 2023-04-09T04:36:34Z

Okay, nice, I think we're pretty close.

Does anything else need to be resolved before we merge this?

BillSchumacher · 2023-04-09T04:37:49Z

I thought it was good before but you are thorough =)

Torantulino · 2023-04-09T04:39:29Z

Gotta look out for all the users!

It's passing all my tests now though.

I have a few more suggestions for tweaks to the memory system now, but they're better suited to their on PR.
Keep and eye out for them and let me know what you think of them.

BillSchumacher · 2023-04-09T04:39:50Z

Will do

BillSchumacher added 2 commits April 6, 2023 22:25

Create an abstract MemoryProviderSingleton class. Pass config instead…

6819799

… of instantiating a new one where used.

Implement redis memory backend.

5a1d9e6

Save redis memory state, with the default being to wipe on start still.

cce7969

Update README with WIPE_REDIS_ON_START setting.

43746b1

BillSchumacher added 6 commits April 7, 2023 01:54

Merge branch 'Torantulino:master' into redis-backend

df28193

Fix README

f016203

Remove unused function.

5d13fb2

Add configurable index key for redis.

14e10c9

Update README

ea6b970

Implement local memory.

cb14c8d

BillSchumacher changed the title ~~Implement Redis Memory backend~~ Implement Local Cache and Redis Memory backend Apr 7, 2023

Refactor memory into factory.

503b58b

Torantulino added the enhancement New feature or request label Apr 8, 2023

Merge branch 'master' of github.com:Torantulino/Auto-GPT into redis-b…

d7ccaf4

…ackend

jkoelker previously approved these changes Apr 8, 2023

View reviewed changes

scripts/config.py Outdated Show resolved Hide resolved

BillSchumacher dismissed jkoelker’s stale review via a34c51b April 8, 2023 01:58

Update scripts/config.py

a34c51b

Co-authored-by: Jason Kölker <jason@koelker.net>

jkoelker previously approved these changes Apr 8, 2023

View reviewed changes

Torantulino previously approved these changes Apr 9, 2023

View reviewed changes

Merge branch 'master' into redis-backend

b63238f

Torantulino dismissed their stale review via b63238f April 9, 2023 01:43

Torantulino added the high priority label Apr 9, 2023

Torantulino requested changes Apr 9, 2023

View reviewed changes

scripts/main.py Outdated Show resolved Hide resolved

Update main.py

2db7f08

Remove pinecone config requirement

Torantulino reviewed Apr 9, 2023

View reviewed changes

scripts/memory/__init__.py Show resolved Hide resolved

Wipe local memory on load

9e139fb

Torantulino reviewed Apr 9, 2023

View reviewed changes

scripts/memory/redismem.py Outdated Show resolved Hide resolved

Memory fixes.

a861dec

Torantulino approved these changes Apr 9, 2023

View reviewed changes

Torantulino merged commit df7ddd4 into Significant-Gravitas:master Apr 9, 2023

This was referenced Apr 9, 2023

Auto-GPT and Pinecone [!!use extreme caution!!] #502

Closed

Save Long-Term-Memory to file on shutdown #9

Closed

alreadydone mentioned this pull request Apr 9, 2023

Pinecone Vectorized Memory #122

Merged

This was referenced Apr 10, 2023

Not creating pinecone index #711

Closed

Adds information on how to use the other cache methods available #794

Merged

swyxio mentioned this pull request Apr 13, 2023

Use Chroma to augment and make LocalCache memory/embeddings pluggable #1027

Closed

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement Local Cache and Redis Memory backend #372

Implement Local Cache and Redis Memory backend #372

BillSchumacher commented Apr 7, 2023 •

edited

Loading

BillSchumacher commented Apr 7, 2023

BillSchumacher commented Apr 7, 2023

aosan commented Apr 7, 2023

BillSchumacher commented Apr 7, 2023

BillSchumacher commented Apr 7, 2023

BillSchumacher commented Apr 7, 2023

dominikdassow commented Apr 8, 2023

supermario-ai commented Apr 8, 2023 •

edited

Loading

supermario-ai commented Apr 8, 2023

supermario-ai commented Apr 8, 2023

jkoelker left a comment •

edited

Loading

BillSchumacher commented Apr 8, 2023

Torantulino commented Apr 8, 2023

ryanpeach commented Apr 9, 2023

Torantulino commented Apr 9, 2023

BillSchumacher commented Apr 9, 2023

Torantulino commented Apr 9, 2023

Torantulino Apr 9, 2023

Torantulino Apr 9, 2023

BillSchumacher Apr 9, 2023

Torantulino Apr 9, 2023

BillSchumacher Apr 9, 2023

BillSchumacher commented Apr 9, 2023

BillSchumacher commented Apr 9, 2023

Torantulino commented Apr 9, 2023

Torantulino commented Apr 9, 2023

BillSchumacher commented Apr 9, 2023

Torantulino commented Apr 9, 2023

BillSchumacher commented Apr 9, 2023

Implement Local Cache and Redis Memory backend #372

Implement Local Cache and Redis Memory backend #372

Conversation

BillSchumacher commented Apr 7, 2023 • edited Loading

Background

Changes

Test Plan

Change Safety

BillSchumacher commented Apr 7, 2023

BillSchumacher commented Apr 7, 2023

aosan commented Apr 7, 2023

BillSchumacher commented Apr 7, 2023

BillSchumacher commented Apr 7, 2023

BillSchumacher commented Apr 7, 2023

dominikdassow commented Apr 8, 2023

supermario-ai commented Apr 8, 2023 • edited Loading

supermario-ai commented Apr 8, 2023

supermario-ai commented Apr 8, 2023

jkoelker left a comment • edited Loading

Choose a reason for hiding this comment

BillSchumacher commented Apr 8, 2023

Torantulino commented Apr 8, 2023

ryanpeach commented Apr 9, 2023

Torantulino commented Apr 9, 2023

BillSchumacher commented Apr 9, 2023

Torantulino commented Apr 9, 2023

Torantulino Apr 9, 2023

Choose a reason for hiding this comment

Torantulino Apr 9, 2023

Choose a reason for hiding this comment

BillSchumacher Apr 9, 2023

Choose a reason for hiding this comment

Torantulino Apr 9, 2023

Choose a reason for hiding this comment

BillSchumacher Apr 9, 2023

Choose a reason for hiding this comment

BillSchumacher commented Apr 9, 2023

BillSchumacher commented Apr 9, 2023

Torantulino commented Apr 9, 2023

Torantulino commented Apr 9, 2023

BillSchumacher commented Apr 9, 2023

Torantulino commented Apr 9, 2023

BillSchumacher commented Apr 9, 2023

BillSchumacher commented Apr 7, 2023 •

edited

Loading

supermario-ai commented Apr 8, 2023 •

edited

Loading

jkoelker left a comment •

edited

Loading