[Docs]: add advanced kubernetes deployment examples

[haitwang-cloud]

Pull Request Description

Add comprehensive documentation for deploying AIBrix on Kubernetes with persistent model caching, security policies, and monitoring integration. Includes YAML examples for namespace, deployment, service, and PVC configuration with detailed usage notes.

Related Issues

Resolves: #[Insert issue number(s)]

Important: Before submitting, please complete the description above and review the checklist below.

---

Contribution Guidelines (Expand for Details)

We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:

Pull Request Title Format

Your PR title should start with one of these prefixes to indicate the nature of the change:

• [Bug]: Corrections to existing functionality
• [CI]: Changes to build process or CI pipeline
• [Docs]: Updates or additions to documentation
• [API]: Modifications to aibrix's API or interface
• [CLI]: Changes or additions to the Command Line Interface
• [Misc]: For changes not covered above (use sparingly)

Note: For changes spanning multiple categories, use multiple prefixes in order of importance.

Submission Checklist

• PR title includes appropriate prefix(es)
• Changes are clearly explained in the PR description
• New and existing tests pass successfully
• Code adheres to project style and best practices
• Documentation updated to reflect changes (if applicable)
• Thorough testing completed, no regressions introduced

By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.

[gemini-code-assist]

Summary of Changes

Hello @haitwang-cloud, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces new, detailed documentation for advanced Kubernetes deployments of AIBrix. It provides a practical YAML example demonstrating how to set up AIBrix with features like persistent model caching, defined security policies, optimized resource allocation, and integrated health monitoring and metrics collection.

Highlights

• New Documentation: I've added a new documentation page, advanced-k8s-examples.rst, providing comprehensive examples for deploying AIBrix on Kubernetes.
• Kubernetes Deployment Example: The new documentation includes a detailed YAML example demonstrating how to deploy a vLLM model (specifically deepseek-ai/DeepSeek-R1-Distill-Llama-8B) on Kubernetes, covering Namespace, Deployment, Service, and PersistentVolumeClaim configurations.
• Persistent Model Caching: The example showcases how to configure a PersistentVolumeClaim (PVC) to provide 30Gi of persistent storage for HuggingFace model caches at /root/.cache/huggingface, ensuring models are not re-downloaded on pod restarts.
• Security and Resource Management: The provided YAML includes configurations for an isolated namespace with a privileged security policy, explicit GPU, CPU, and memory resource limits and requests, and shared memory (/dev/shm) setup.
• Health Checks and Monitoring: The deployment example integrates liveness and readiness probes for robust health monitoring, along with Prometheus annotations on the Service for metrics integration and service discovery.
• Documentation Integration: I've updated installation.rst to link to the new advanced Kubernetes examples, making them discoverable within the existing documentation structure.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[haitwang-cloud]

/gemini review

[gemini-code-assist]

Code Review

The pull request adds documentation for advanced Kubernetes deployment examples, including YAML configurations for deploying AIBrix with persistent model caching, security policies, and monitoring integration. The documentation provides valuable insights into optimizing resource configuration and setting up health monitoring.

[gemini-code-assist]

Code Review

The pull request adds documentation for advanced Kubernetes deployment examples for AIBrix, including YAML configurations for namespace, deployment, service, and PVC. The documentation covers persistent model caching, security policies, and monitoring integration. The review identifies potential security concerns with the privileged security policy and suggests improvements to resource limit definitions, probe configurations, and storage class guidance.

[haitwang-cloud]

Fixing the format error in

/aibrix/docs/source/features/gateway-plugins.rst:330: ERROR: Error in "code-block" directive:
maximum 1 argument(s) allowed, 6 supplied.

.. code-block:: bash
    kubectl get pods -n envoy-gateway-system

    NAME                                                      READY   STATUS    RESTARTS   AGE
    envoy-aibrix-system-aibrix-eg-903790dc-84ccfcbc6b-hw2lq   2/2     Running   0          13m
    envoy-gateway-7c7659ffc9-rvm5s                            1/1     Running   0          16m

    kubectl logs envoy-aibrix-system-aibrix-eg-903790dc-84ccfcbc6b-hw2lq -n envoy-gateway-system [docutils]
/Users/I742968/gpu-related/aibrix/docs/source/features/gateway-plugins.rst:339: ERROR: Error in "code-block" directive:
maximum 1 argument(s) allowed, 6 supplied.

.. code-block:: bash
    kubectl get pods -n aibrix-system

    NAME                                        READY   STATUS             RESTARTS   AGE
    aibrix-controller-manager-fb4495448-j9k6g   1/1     Running            0          22m
    aibrix-gateway-plugins-6bd9fcd5b9-2bwpr     1/1     Running            0          22m
    aibrix-gpu-optimizer-df9db96c8-2fctd        1/1     Running            0          22m
    aibrix-kuberay-operator-5bf4985d86-7g4tz    1/1     Running            0          22m
    aibrix-metadata-service-9d4cd7f77-mq7tr     1/1     Running            0          22m
    aibrix-redis-master-7d6b77c794-bcqxc        1/1     Running            0          22m

    kubectl logs aibrix-gateway-plugins-6bd9fcd5b9-2bwpr -n aibrix-system [docutils]

/aibrix/docs/source/getting_started/installation/advanced-k8s-examples.rst:3: WARNING: Title overline too short.

=========================
Advanced Kubernetes Examples
========================= [docutils]
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
copying assets... 
copying static files... done
copying extra files... done
copying assets: done
writing output... [100%] production/observability

/aibrix/docs/source/getting_started/quickstart.rst:111: WARNING: Lexing literal_block '# multiturn conversation\npackage main\n\nimport (\n    "context"\n\n    "github.com/openai/openai-go"\n    "github.com/openai/openai-go/option"\n)\n\nfunc main() {\n    client := openai.NewClient(\n        option.WithBaseURL("http://${ENDPOINT}:8888/v1"),\n        option.WithAPIKey("OPENAI_API_KEY"),\n        option.WithHeader("routing-strategy", "prefix-cache"),\n    )\n    chatCompletion, _ := client.Chat.Completions.New(context.TODO(), openai.ChatCompletionNewParams{\n        Messages: []openai.ChatCompletionMessageParamUnion{\n            openai.SystemMessage("You are a helpful assistant."),\n            openai.UserMessage("What is the capital of California?"),\n        },\n        Model: "deepseek-r1-distill-llama-8b",\n    })\n    println(chatCompletion.Choices[0].Message.Content)\n\n    chatCompletion, _ = client.Chat.Completions.New(context.TODO(), openai.ChatCompletionNewParams{\n        Messages: []openai.ChatCompletionMessageParamUnion{\n            openai.SystemMessage("You are a helpful assistant."),\n            openai.UserMessage("What is the capital of California?"),\n            openai.AssistantMessage(chatCompletion.Choices[0].Message.Content),\n            openai.UserMessage("What is the largest county of california?"),\n        },\n        Model: "deepseek-r1-distill-llama-8b",\n    })\n    println(chatCompletion.Choices[0].Message.Content)\n}' as "go" resulted in an error at token: '#'. Retrying in relaxed mode. [misc.highlighting_failure]

[haitwang-cloud]

/gemini review

[gemini-code-assist]

Code Review

The pull request adds documentation for advanced Kubernetes deployment examples, including YAML configurations for deploying AIBrix with persistent model caching, security policies, and monitoring integration. The documentation is well-structured and provides valuable information for users looking to deploy AIBrix on Kubernetes. However, some minor improvements could be made to enhance clarity and usability.

[Jeffwan]

looks good at this moment. I plan to add an example section to host such examples.