[3/N] Achieve fault tolerance at the DP level

[ympcMark]

Motivation

The previous work of this PR only implemented fault tolerance at the EP level, but not at the DP level; we achieved this functionality by maintaining worker information through communication between the Scheduler, TokenizerManager, and DataParallelController.

Modifications

io_struct.py: Define scheduler status information

scheduler.py: Send status information to TokenizerManager.

tokenizer_manager.py: Write a handler function to accept this type of information and send information to DataParallelController.

data_parallel_controller.py: Write a handler function to accept this type of information and maintain worker information.

Accuracy Tests

added a new unit test test/srt/ep/test_mooncake_ep_small.py

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[HanHan009527]

I'm thinking, would it be more reasonable to rename the elastic ep module to elstic and group together concepts like _TP_ACTIVE_RANKS and _TP_ACTIVE_RANKS_CPU as much as possible?

[HanHan009527]

Make it less intrusive to the original logic

[UNIDY2002]

Sounds reasonable