[BUG]: Cannot find module 'colossalai.kernel.cuda_native' when run sequence_parallel example

[ztorchan]

🐛 Describe the bug

I noticed that after v0.3.5 of ColossalAI, the colossalai.kernel.cuda_native module seems to have been removed.

https://github.com/hpcaitech/ColossalAI/tree/8e412a548e5366d1c42bcf386bd185091bd0c280/colossalai/kernel

However, the statement from colossalai.kernel.cuda_native import LayerNorm still exists in examples/tutorial/sequence_parallel/model/layers/bert_layer.py,

ColossalAI/examples/tutorial/sequence_parallel/model/layers/bert_layer.py

Line 4 in 8e412a5

from colossalai.kernel.cuda_native import LayerNorm

which leads to errors.

ModuleNotFoundError: No module named 'colossalai.kernel.cuda_native'

Is it normal?

Environment

python 3.10.10
cuda 11.6.1
torch 1.12.1

[Edenzzzz]

This is deprecated and yes we should remove it. @Courtesy-Xs could you point to where this kernel is located now?

[ztorchan]

@Edenzzzz I have changed some code to fix the bug. It can run on my machine now. Please check the PR.

[Edenzzzz]

Thanks for your attention. Though tutorials are part of the legacy API that we are deprecating, and I've added a similar fix for other tutorials to make them runnable.

[ztorchan]

Thanks. I will close the #5556