add launch_server and StartArgs

[sufubao]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @sufubao, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the server's initialization and argument handling by introducing a dedicated launch_server function and standardizing the use of the StartArgs data class. This change not only cleans up the main execution flow but also expands the server's configurability by adding numerous new parameters to StartArgs, allowing for more flexible and feature-rich deployments.

Highlights

• Server Launch Refactoring: The main server startup logic in api_server.py has been encapsulated into a new launch_server function, improving modularity and readability.
• Standardized Argument Handling: The StartArgs data class is now consistently used for type hinting and initializing server arguments across api_server.py and api_start.py, enhancing type safety and clarity.
• Expanded Server Configuration: The StartArgs class has been extended with several new configuration parameters, offering more granular control over HTTP server workers, shared memory warnings, data parallelism balancing, custom all-gather/all-reduce, fused shared experts, MPS, multinode routing, scheduling intervals, and dynamic prompt caching.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request refactors the server startup logic by introducing a launch_server function and centralizing configuration in the StartArgs dataclass. This is a positive change for code structure and maintainability. However, I've found a critical issue in api_server.py that will prevent the server from starting due to incorrect argument unpacking. I've also noted a minor point about including a deprecated argument in the StartArgs dataclass, which could be confusing.

[gemini-code-assist]

The argparse.Namespace object returned by parser.parse_args() cannot be unpacked directly into a dataclass constructor using **args. This will raise a TypeError because the keys of the namespace are not strings. You should convert the namespace to a dictionary using vars(args) before unpacking.

Suggested change

	launch_server(StartArgs(**args))
	launch_server(StartArgs(**vars(args)))