Great work. quick question on roadmap ETA?

[darkman111a]

Just wondering when you were planning on implementing BitLinear layer to use 1-bit weights and custom cuda kernel for 1-bit weight? super thirsty for the code ha. Appreciate you.

[Beomi]

Currently I'm investigating on it. plan to make it compatible with Bitsandbytes.
Since I'm not familiar with CUDA or C++, it'll takes some time,
so currently I could not guarantee the ETA. 😂

[DewEfresh]

you may want to check out https://github.com/IST-DASLab/qmoe. They created some custom cuda functions for sub 1-bit weights.

[Beomi]

you way want to check out https://github.com/IST-DASLab/qmoe. They created some custom cuda functions for sub 1-bit weights.

Thanks for information!
I'm looking on it 😉

[darkman111a]

Appreciate you. Stay Blessed! This might assist you as well -> https://github.com/brycexu/BNN_Kernel