Repository navigation
How to use this most efficiently? #45
Description
Hi,
I want to store 214 bits in a BitSet.
When i do that it's data array is 10 elements. That's 10x 32bit int (320 bit).
Ideally it would use the bare minimum in bit overhead. If that were to be done in 32 bit ints then 7 ints (224 bit) would suffice.
More ideal would be an array of chars (8 bit each) where i'd need 27 chars giving me 216 bits of room.
Are there arguments i can set in this class to get that more ideal behavior (ints or chars, both work)?
Activity
infusion commentedon Sep 14, 2022
Hi
currently you can't control that behavior, since the library is optimized for speed, not memory. The 10 elements are a reasonable pre-allocation at the beginning to save re-allocations in most basic usecases < 320 bit. Especially when you narrow it down the way you're asking for you can't handle big jumps properly, such as
.set(2),.set(1000). In your desired way you would increase the size linearly, instead of pre-allocating larger ranges.Do you have a use case where 3*4 byte really make such a difference?
markg85 commentedon Sep 15, 2022
Hi,
Personally i'd argue that using a bitset most often implies one is going for memory efficiency above performance.
I do have a usecase where this really matters though it's very much niche.
I'm just playing with a site to show crypto prices. I'm using the crypto.com ticker API for that which gives me that number of 214 (that many different cryptos). In my specific case i get enough data from the ticker to send an update to the browser every .05 seconds. I'm already limiting the data i send a lot to the bare minimum but that's still a list of 214 entries with the difference between the previous message.
This brings down data usage per 0.5 seconds to ~300 bytes.
Now if i have a bitset that tells me which entries changed then i only need to send:
So yeah, this very niche case, does in fact save quite a bit of data if it were possible with BitSet.
markg85 commentedon Sep 15, 2022
So, a very quick experiment with https://github.com/Callidon/bloom-filters/blob/master/src/bloom/bit-set.ts (from the bloom-filters package, the bitset is just for internal purposes).
That is memory efficient and it does in fact turn out that using such a structure gives me 0.5 second data blobs between 40-100 bytes. The API of that class is weird though... Oh well, works for my needs.
Could you perhaps consider adding a settable flag to your class to choose between memory efficiency and performance?