[autoparallel] distinguish different parallel strategies

[YuliangLiu0306]

📌 Checklist before creating the PR

• I have created an issue for this PR for traceability
• The title follows the standard format: [doc/gemini/tensor/...]: A concise description
• I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Distinguish different parallel strategies in strategy_generator. For example, we will generate dp only strategy if solver_preference is 'dp', tp only strategy if solver_preference is 'tp', mix strategies if solver_preference is 'standard'.

As the strategies order has been changed, the unit tests adapts to the lastest linear generator. Why not just remove the index? To make sure the following generators correct, we have to check the strategies one by one.

💥 Checklist before requesting a review

• I have linked my PR to an issue (instruction)
• My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
• I have performed a self-review of my code
• I have added thorough tests.
• I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

• 🌝 Yes, I do.
• 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

[github-actions]

The code coverage for the changed files is 18%.

Click me to view the complete report

Name                                                                                                 Stmts   Miss  Cover
------------------------------------------------------------------------------------------------------------------------
colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py                                   117     94    20%
colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py               390    321    18%
tests/test_auto_parallel/test_tensor_shard/test_gpt/test_runtime_with_gpt_modules.py                   133     94    29%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/test_permute_and_transpose_handler.py     245    213    13%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/test_softmax_handler.py                   126     98    22%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/test_split_handler.py                     186    154    17%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/test_view_handler.py                      185    154    17%
------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                 1382   1128    18%

[FrankLeeeee]

Hi, @YuliangLiu0306 remember to write an issue first and link your pull request to the issue for back-traceability. This can better help others understand your PR and follow up the feature progress.

[github-actions]

The code coverage for the changed files is 18%.

Click me to view the complete report

Name                                                                                                 Stmts   Miss  Cover
------------------------------------------------------------------------------------------------------------------------
colossalai/auto_parallel/tensor_shard/node_handler/linear_handler.py                                   117     94    20%
colossalai/auto_parallel/tensor_shard/node_handler/strategy/matmul_strategy_generator.py               390    321    18%
tests/test_auto_parallel/test_tensor_shard/test_gpt/test_runtime_with_gpt_modules.py                   133     94    29%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/test_permute_and_transpose_handler.py     245    213    13%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/test_softmax_handler.py                   126     98    22%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/test_split_handler.py                     186    154    17%
tests/test_auto_parallel/test_tensor_shard/test_node_handler/test_view_handler.py                      185    154    17%
------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                 1382   1128    18%