compress_block_sliding_stride

[Mr-Grin]

#23

[lucidrains]

cool! tests are not passing though 🤔

was this LLM assisted? some of the code looks strange

[Mr-Grin]

hi I think this is because I didn't update the test on alternate MLP, so it missing the newly added argument compress block sliding stride. It passed on NSA test though.

Part of the code is written by LLM, I think I shouldn't have use it because LLM missed to modify on some part of the code so I spend sometime debugging it.

I will add the argument on alternative mlp later this day

[lucidrains]

@Mr-Grin it is close! as long as tests pass and train.py script works, we can merge!

[Mr-Grin]

@lucidrains it passed the GitHub automatic check, train.py takes 5 hours to run on my computer, should I run all the way through it?

[lucidrains]

@Mr-Grin no just the first 1k steps will do! this is great, thank you so much!