[Enhancement]: Management WebUI for enhanced Milvus observability.

[jaime0815]

Is there an existing issue for this?

• I have searched the existing issues

What would you like to be added?

Goal

Simplify Milvus operations.

Value

Enhance system observability for quicker issue resolution.

Tools Comparison

Feature	Management WebUI	Birdwatcher	Attu
Access	Web UI	Command-line tool	GUI
Target User	Operations/Development	Operations	Development
Integration	Embedded in Milvus	Standalone	Standalone
Dependency	Requires healthy Milvus	Requires healthy Milvus/ETCD	Requires healthy Milvus
Functionality	System environment and runtime metrics	Querying and modifying ETCD metadata, calling some MIlvus APIs	Database management and operations

Demo

Access the local standalone Milvus web UI by address: http://localhost:9091/webui/

Screen.Recording.2024-09-30.at.17.24.28.mov

TODO:

• Basic Web UI Server

Cluster Metrics

• Display building information.
• Display deployment information.
• Track details of connected clients.

Server Metrics

• Monitor sealed and growing segments on QN and DN.
• Track subscribed channels on QN and DN.
• Count processed msgpack on QN and DN.
• Server health status.

Database Metrics

• List all databases (name, creation time).

Collection Metrics

• Gather basic information (partitions, channels, replicas, index tables).
• Monitor row count in collections.
• Track read and write operations.
• List segments.
• Evaluate the health status of collections.

Replica Metrics

• replica list
• Track QPS (Queries Per Second).

Resource group metrics

• resource groups list

Channel Metrics

• Pchannels list.
• Count subscriptions for each Pchannel.
• Track producers on each Pchannel.
• Monitor timetick, DDL, and DML messages per Pchannel.
• List Vchannels for each Pchannel.
• Capture the latest checkpoint MessageID for each Vchannel.
• Track the latest ts of the message for each Vchannel.
• Monitor messages consumed and produced per second for each Vchannel.

Task Metrics

• Track index-building tasks (index name, collection name, index ID, collection ID, type, field).
• Monitor balance segment tasks (source/dest, segment ID, row count, status).
• Track balance channel tasks (source/dest, channel name).
• Monitor flush tasks (collection, partition, segment ID, type, log stats, row count).
• Track compaction tasks (source/dest segment ID, download/merge time).
• Monitor load/release tasks for collections, partitions, and segments.
• Track bulk insert tasks.

DevOps

• Update configurations.
• Collect pprof data for performance profiling.
• Enable trace logging for debugging.
• Visualize memory variables.
• Show audit logs of actions taken.
• Allow manual execution of flush, compaction, and balance tasks.
• Add intelligent diagnosis features.

Why is this needed?

No response

Anything else?

No response

[tangkunyin]

However, I tried http://localhost:9091/webui/ but got 404...

My services under docker container were all running well,I have refered to the official document, but no useful things were found

Is there anyone who can tell me how to use webui? many thanks

[xiaofan-luan]

However, I tried http://localhost:9091/webui/ but got 404...

My services under docker container were all running well,I have refered to the official document, but no useful things were found

Is there anyone who can tell me how to use webui? many thanks

1. this is only worked for Milvus 2.5 or above.
2. you will need to expose 9091 port, which already work on your environment.

[tangkunyin]

@xiaofan-luan Yes, that's true. It would be better to give a description of the official documentation 🙏🏻