Skip to content
This repository has been archived by the owner on Jul 30, 2024. It is now read-only.

how to use bloomfilter set expire time #37

Closed
luoqishuai opened this issue Aug 19, 2019 · 3 comments
Closed

how to use bloomfilter set expire time #37

luoqishuai opened this issue Aug 19, 2019 · 3 comments

Comments

@luoqishuai
Copy link

I want to use bloomfilter in scrapy_redis. scrapy_redis has a large number of url need to filter.
If i use the bloomfilter for a few weeks. Linux memory will boom.So i want use bloomfilter to filter 7 days(or other time) url.What shou i do? Can you give me some adices?

@luoqishuai
Copy link
Author

If you don't understand or have no free time .Please tell me. Thanks.

@kleinron
Copy link

I think you need some kind of aging Bloom filter, as described here.
I'm not sure if this repo contains such implementation.

@luoqishuai
Copy link
Author

Thank you for your help.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants