Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/state: replace fastcache code cache with gc-friendly structure #26092

Merged
merged 6 commits into from
Nov 9, 2022

Conversation

holiman
Copy link
Contributor

@holiman holiman commented Nov 2, 2022

Initial poc, one way to replace fastcache with a pretty simple LRU which does not require explicit closing.

This is an alternative to #26071

Copy link

@a1love-web a1love-web left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial poc, one way to replace fastcache with a pretty simple LRU which does not require explicit closing.

@holiman holiman marked this pull request as ready for review November 3, 2022 08:14
common/lru/blob_lru.go Outdated Show resolved Hide resolved
@PlasmaPower
Copy link
Contributor

Should this (or something else) be used for the cleans cache too? That's also not currently being freed:

cleans *fastcache.Cache // GC friendly memory cache of clean node RLPs

common/lru/blob_lru.go Outdated Show resolved Hide resolved
common/lru/blob_lru.go Outdated Show resolved Hide resolved
Copy link

@a1love-web a1love-web left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link

@tsahee tsahee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree this direction is better.
This is a reminder that fastcache is also used in trie.Database (cleans) with the same memory leak.
It's also used in snapshot's disklayer.cache - this should probably be cleaned up as well though I'm not certain if a memleak can be created from snapshot on normal operation.

// Get looks up a key's value from the cache.
func (c *SizeConstrainedLRU) Get(key common.Hash) []byte {
c.lock.RLock()
defer c.lock.RUnlock()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick, please add a blank line below, make eyes feel better.

c.lock.RLock()
defer c.lock.RUnlock()
if v, ok := c.lru.Get(key); ok {
return []byte(v.(string))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not we always accept []byte type value instead of converting it all the time? Code is []byte, we can set/get it into/from cache directly without any conversion.

If you think the passed byte slice might be "mutable", we can just add a comment that the passed value must not be changed.

@rjl493456442
Copy link
Member

@tsahee It's another story, we are aware of it! Snapshot and TrieDB are all should be singleton in Geth and only released when node is shutdown. But we might abuse it somewhere, will check it.

@tsahee
Copy link

tsahee commented Nov 7, 2022

@rjl493456442 Hi!
As far as I can tell TrieDB is created whenever a stateDatabase is created. Specifically, this line in StateAtBlock will create a fastcache for TrieDB's "cleans" and leak memory.

@rjl493456442
Copy link
Member

@tsahee thanks for the reminder, regard trie.Database, I think we should add this Close function to it and explicitly close all opened trie database after usage. I will think how to implement it.

@holiman holiman added this to the 1.11.0 milestone Nov 8, 2022
@holiman holiman merged commit 5fded04 into ethereum:master Nov 9, 2022
lochjin pushed a commit to lochjin/go-ethereum that referenced this pull request Nov 10, 2022
…thereum#26092)

This PR replaces fastcache with a pretty simple LRU which does not require explicit closing.

(cherry picked from commit 5fded04)
lochjin pushed a commit to lochjin/go-ethereum that referenced this pull request Nov 30, 2022
…thereum#26092)

This PR replaces fastcache with a pretty simple LRU which does not require explicit closing.

(cherry picked from commit 5fded04)
shekhirin pushed a commit to shekhirin/go-ethereum that referenced this pull request Jun 6, 2023
…thereum#26092)

This PR replaces fastcache with a pretty simple LRU which does not require explicit closing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants