Skip to content

Commit

Permalink
[MINOR] improve(docs): update the doc of the web (apache#2907)
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

Add description for creating fileset and messaging catalog, and add some
tips.

### Why are the changes needed?

Fix: apache#2100

### Does this PR introduce _any_ user-facing change?

N/A

### How was this patch tested?

N/A
  • Loading branch information
ch3yne authored and diqiu50 committed Jun 13, 2024
1 parent 92f849b commit b7ff5f4
Show file tree
Hide file tree
Showing 5 changed files with 96 additions and 65 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/webui/fileset-details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/webui/topic-details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
161 changes: 96 additions & 65 deletions docs/webui.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,20 @@ title: 'Gravitino web UI'
slug: /webui
keyword: webui
last_update:
date: 2024-02-04
date: 2024-04-12
author: ch3yne
license: 'Copyright 2023 Datastrato Pvt Ltd. This software is licensed under the Apache License version 2.'
---

import Image from '@theme/IdealImage'
import Tabs from '@theme/Tabs'
import TabItem from '@theme/TabItem'

This document primarily outlines how users can manage metadata within Gravitino using the web UI, the graphical interface is accessible through a web browser as an alternative to writing code or using the REST interface.

Currently, you can integrate [OAuth settings](security.md) to view, add, modify, and delete metalakes, create catalogs, and view catalogs, schemas, and tables, among other functions.
Currently, you can integrate [OAuth settings](./security.md) to view, add, modify, and delete metalakes, create catalogs, and view catalogs, schemas, and tables, among other functions.

[Build](how-to-build.md#quick-start) and [deploy](getting-started.md#getting-started-locally) the Gravitino Web UI and open it in a browser at `http://<gravitino-host>:<gravitino-port>`, by default is [http://localhost:8090](http://localhost:8090).
[Build](./how-to-build.md#quick-start) and [deploy](./getting-started.md#getting-started-locally) the Gravitino Web UI and open it in a browser at `http://<gravitino-host>:<gravitino-port>`, by default is [http://localhost:8090](http://localhost:8090).

## Initial page

The web UI homepage displayed in Gravitino depends on the configuration parameter for OAuth mode, see the details in [Security](security.md).
The web UI homepage displayed in Gravitino depends on the configuration parameter for OAuth mode, see the details in [Security](./security.md).

Set parameter for `gravitino.authenticator`, [`simple`](#simple-mode) or [`oauth`](#oauth-mode). Simple mode is the default authentication option.

Expand All @@ -38,7 +34,7 @@ gravitino.authenticator = simple

Set the configuration parameter `gravitino.authenticator` to `simple`, and the web UI displays the homepage (Metalakes).

![webui-metalakes-simple](assets/webui/metalakes-simple.png)
![webui-metalakes-simple](./assets/webui/metalakes-simple.png)

At the top-right, the UI displays the current Gravitino version.

Expand All @@ -52,13 +48,20 @@ gravitino.authenticator = oauth

Set the configuration parameter `gravitino.authenticator` to `oauth`, and the web UI displays the login page.

![webui-login-with-oauth](assets/webui/login-with-oauth.png)
:::caution
If both `OAuth` and `HTTPS` are set, due to the different security permission rules of various browsers, to avoid cross-domain errors,
it is recommended to use the [Chrome](https://www.google.com/chrome/) browser for access and operation.

Such as Safari need to enable the developer menu, and select `Disable Cross-Origin Restrictions` from the develop menu.
:::

![webui-login-with-oauth](./assets/webui/login-with-oauth.png)

1. Enter the values corresponding to your specific configuration. For detailed instructions, please refer to [Security](security.md).
1. Enter the values corresponding to your specific configuration. For detailed instructions, please refer to [Security](./security.md).

2. Clicking on the `LOGIN` button takes you to the homepage.

![webui-metalakes-oauth](assets/webui/metalakes-oauth.png)
![webui-metalakes-oauth](./assets/webui/metalakes-oauth.png)

At the top-right, there is an icon button that takes you to the login page when clicked.

Expand All @@ -68,47 +71,47 @@ At the top-right, there is an icon button that takes you to the login page when
### Metalake

#### [Create metalake](getting-started.md#using-rest-to-interact-with-gravitino)
#### [Create metalake](./getting-started.md#using-rest-to-interact-with-gravitino)

On the homepage, clicking on the `CREATE METALAKE` button displays a dialog to create a metalake.

![create-metalake-dialog](assets/webui/create-metalake-dialog.png)
![create-metalake-dialog](./assets/webui/create-metalake-dialog.png)

Creating a metalake needs these fields:

1. **Name**(**_required_**): the name of the metalake.
2. **Comment**(_optional_): the comment of the metalake.
3. **Properties**(_optional_): Click on the `ADD PROPERTY` button to add custom properties.

![metalake-list](assets/webui/metalake-list.png)
![metalake-list](./assets/webui/metalake-list.png)

There are 3 actions you can perform on a metalake.

![metalake-actions](assets/webui/metalake-actions.png)
![metalake-actions](./assets/webui/metalake-actions.png)

#### Show metalake details

Clicking on the action icon <Icon icon='bx:show-alt' fontSize='24' /> in the table cell.

You can see the detailed information of this metalake in the drawer component on the right.

![metalake-details](assets/webui/metalake-details.png)
![metalake-details](./assets/webui/metalake-details.png)

#### Edit metalake

Clicking on the action icon <Icon icon='mdi:square-edit-outline' fontSize='24' /> in the table cell.

Displays the dialog for modifying fields of the selected metalake.

![create-metalake-dialog](assets/webui/create-metalake-dialog.png)
![create-metalake-dialog](./assets/webui/create-metalake-dialog.png)

#### Delete metalake

Clicking on the action icon <Icon icon='mdi:delete-outline' fontSize='24' color='red' /> in the table cell.

Displays a confirmation dialog, clicking on the `SUBMIT` button deletes this metalake.

![delete-metalake](assets/webui/delete-metalake.png)
![delete-metalake](./assets/webui/delete-metalake.png)

### Catalog

Expand All @@ -118,45 +121,50 @@ If this is the first time, it shows no data until after creating a catalog.

Clicking on the left arrow icon button <Icon icon='mdi:arrow-left' fontSize='24' color='#6877ef' /> takes you to the metalake page.

![metalake-catalogs](assets/webui/metalake-catalogs.png)
![metalake-catalogs](./assets/webui/metalake-catalogs.png)

Clicking on the Tab - `DETAILS` views the details of the catalog on the metalake catalogs page.

![metalake-catalogs-details](assets/webui/metalake-catalogs-details.png)
![metalake-catalogs-details](./assets/webui/metalake-catalogs-details.png)

On the left side of the page is a tree list.
On the left side of the page is a tree list, and the icons of the catalog correspond to their type and provider.

- Catalog <Icon icon='bx:book' fontSize='24px' />
- Schema <Icon icon='bx:coin-stack' fontSize='24px' />
- Table <Icon icon='bx:table' fontSize='24px' />

![tree-view](assets/webui/tree-view.png)
![tree-view](./assets/webui/tree-view.png)

Hover your mouse over the corresponding icon to the data changes to a reload icon <Icon icon='mdi:reload' fontSize='24px' />. Click on this icon to reload the currently selected data.

![tree-view-reload-catalog](assets/webui/tree-view-reload-catalog.png)
![tree-view-reload-catalog](./assets/webui/tree-view-reload-catalog.png)

#### Create catalog

Clicking on the `CREATE CATALOG` button displays the dialog to create a catalog.

![create-catalog](assets/webui/create-catalog.png)
![create-catalog](./assets/webui/create-catalog.png)

Creating a catalog requires these fields:

1. **Catalog name**(**_required_**): the name of the catalog
2. **Type**(**_required_**): the default value is `relational`
3. **Provider**(**_required_**): `hive`/`iceberg`/`mysql`/`postgresql`
2. **Type**(**_required_**): `relational`/`fileset`/`messaging`, the default value is `relational`
3. **Provider**(**_required_**):
1. Type `relational` - `hive`/`iceberg`/`mysql`/`postgresql`/`doris`
2. Type `fileset` - `hadoop`
3. Type `messaging` - `kafka`
4. **Comment**(_optional_): the comment of this catalog
5. **Properties**(**each `provider` must fill in the required property fields specifically**)

##### Providers

> Required properties in various providers
###### 1. Type `relational`

<Tabs>
<TabItem value='hive' label='Hive'>
Follow the [Apache Hive catalog](apache-hive-catalog) document
Follow the [Apache Hive catalog](./apache-hive-catalog.md) document.

<Image img={require('./assets/webui/props-hive.png')} style={{ width: 480 }} />

Expand All @@ -166,7 +174,7 @@ Creating a catalog requires these fields:

</TabItem>
<TabItem value='iceberg' label='Iceberg'>
Follow the [Lakehouse Iceberg catalog](lakehouse-iceberg-catalog) document
Follow the [Lakehouse Iceberg catalog](./lakehouse-iceberg-catalog.md) document.

the parameter `catalog-backend` provides two values: `hive`, and `jdbc`.

Expand Down Expand Up @@ -197,7 +205,7 @@ Creating a catalog requires these fields:

</TabItem>
<TabItem value='mysql' label='MySQL'>
Follow the [JDBC MySQL catalog](jdbc-mysql-catalog) document
Follow the [JDBC MySQL catalog](./jdbc-mysql-catalog.md) document.

<Image img={require('./assets/webui/props-mysql.png')} style={{ width: 480 }} />

Expand All @@ -210,7 +218,7 @@ Creating a catalog requires these fields:

</TabItem>
<TabItem value='postgresql' label='PostgreSQL'>
Follow the [JDBC PostgreSQL catalog](jdbc-postgresql-catalog) document
Follow the [JDBC PostgreSQL catalog](./jdbc-postgresql-catalog) document.

<Image img={require('./assets/webui/props-pg.png')} style={{ width: 480 }} />

Expand All @@ -223,40 +231,70 @@ Creating a catalog requires these fields:
|jdbc-database|e.g. `pg_database` |

</TabItem>
<TabItem value='doris' label='Doris'>
Follow the [JDBC Doris catalog](jdbc-doris-catalog) document
<TabItem value='doris' label='Doris'>
Follow the [JDBC Doris catalog](./jdbc-doris-catalog.md) document.

<Image img={require('./assets/webui/props-doris.png')} style={{ width: 480 }} />

|Key |Description |
|-------------|-----------------------------------------------------|
|jdbc-driver |e.g. `com.mysql.jdbc.Driver` |
|jdbc-url |e.g. `jdbc:mysql://localhost:9030` |
|jdbc-user |The JDBC user name |
|jdbc-password|The JDBC password |
|Key |Description |
|-------------|---------------------------------------------------------------------|
|jdbc-driver |JDBC URL for connecting to the database. e.g. `com.mysql.jdbc.Driver`|
|jdbc-url |e.g. `jdbc:mysql://localhost:9030` |
|jdbc-user |The JDBC user name |
|jdbc-password|The JDBC password |

</TabItem>
</Tabs>

:::tip
Due to the current limitation of the web interface, which only allows for viewing, the functionality to create or modify schema, tables, or filesets is not available. Please refer to the [documentation](./manage-fileset-metadata-using-gravitino.md) to use the REST API for these operations.
:::

###### 2. Type `fileset`

<Tabs>
<TabItem value='hadoop' label='Hadoop'>
Follow the [Hadoop catalog](./hadoop-catalog.md) document.

<Image img={require('./assets/webui/create-fileset-hadoop-catalog-dialog.png')} style={{ width: 480 }} />

</TabItem>
</Tabs>

###### 3. Type `messaging`

<Tabs>
<TabItem value='kafka' label='Kafka'>
Follow the [Kafka catalog](./kafka-catalog.md) document.

<Image img={require('./assets/webui/create-messaging-kafka-catalog-dialog.png')} style={{ width: 480 }} />

| Key | Description |
| ----------------- | ----------------------------------------------------------------------------------------- |
| bootstrap.servers | The Kafka broker(s) to connect to, allowing for multiple brokers by comma-separating them |

</TabItem>
</Tabs>

After verifying the values of these fields, clicking on the `CREATE` button creates a catalog.

![created-catalog](assets/webui/created-catalog.png)
![created-catalog](./assets/webui/created-catalog.png)

#### Show catalog details

Clicking on the action icon <Icon icon='bx:show-alt' fontSize='24' /> in the table cell.

You can see the detailed information of this catalog in the drawer component on the right.

![show-catalog-details](assets/webui/show-catalog-details.png)
![show-catalog-details](./assets/webui/show-catalog-details.png)

#### Edit catalog

Clicking on the action icon <Icon icon='mdi:square-edit-outline' fontSize='24' /> in the table cell.

Displays the dialog for modifying fields of the selected catalog.

![update-catalog](assets/webui/update-catalog.png)
![update-catalog](./assets/webui/update-catalog.png)

Only the `name`, `comment`, and custom fields in `properties` can be modified, other fields such as `type`, `provider`, and default fields in `properties` cannot be modified.

Expand All @@ -268,33 +306,23 @@ Clicking on the action icon <Icon icon='mdi:delete-outline' fontSize='24' color=

Displays a confirmation dialog, clicking on the SUBMIT button deletes this catalog.

![delete-catalog](assets/webui/delete-catalog.png)
![delete-catalog](./assets/webui/delete-catalog.png)

### Schema

Under Construction...

- [x] View
- [ ] Create
- [ ] Edit
- [ ] Delete

Design draft preview:

![demo-tables](assets/webui/demo-tables.png)
![demo-tables](./assets/webui/demo-tables.png)

### Table

Under Construction...
![demo-columns](./assets/webui/demo-columns.png)

- [x] View
- [ ] Create
- [ ] Edit
- [ ] Delete
### Fileset

Design draft preview:
![fileset-details](./assets/webui/fileset-details.png)

![demo-columns](assets/webui/demo-columns.png)
### Topic

![topic-details](./assets/webui/topic-details.png)

## Feature capabilities

Expand All @@ -304,21 +332,24 @@ Design draft preview:
| Catalog | _`View`_ &#10004; / _`Create`_ &#10004; / _`Edit`_ &#10004; / _`Delete`_ &#10004; |
| Schema | _`View`_ &#10004; / _`Create`_ &#10008; / _`Edit`_ &#10008; / _`Delete`_ &#10008; |
| Table | _`View`_ &#10004; / _`Create`_ &#10008; / _`Edit`_ &#10008; / _`Delete`_ &#10008; |
| Fileset | _`View`_ &#10004; / _`Create`_ &#10008; / _`Edit`_ &#10008; / _`Delete`_ &#10008; |
| Topic | _`View`_ &#10004; / _`Create`_ &#10008; / _`Edit`_ &#10008; / _`Delete`_ &#10008; |

## E2E test

End-to-end testing for web frontends is conducted using the [Selenium](https://www.selenium.dev/documentation/) testing framework, which is Java-based.

Test cases can be found in the project directory: `integration-test/src/test/java/com/datastrato/gravitino/integration/test/web/ui`, where the `pages` directory is designated for storing definitions of frontend elements, among others.
The root directory contains the actual steps for the test cases.

:::tip
While writing test cases, running them in a local environment may not pose any issues.
While writing test cases, running them in a local environment may not pose any issues.

However, due to the limited performance capabilities of GitHub Actions, scenarios involving delayed DOM loading—such as the time taken for a popup animation to open—can result in test failures.
However, due to the limited performance capabilities of GitHub Actions, scenarios involving delayed DOM loading—such as the time taken for a popup animation to open—can result in test failures.

To circumvent this issue, it is necessary to manually insert a delay operation, for instance, by adding such as `Thread.sleep(sleepTimeMillis)`.
To circumvent this issue, it is necessary to manually insert a delay operation, for instance, by adding such as `Thread.sleep(sleepTimeMillis)`.

This ensures that the test waits for the completion of the delay animation before proceeding with the next operation, thereby avoiding the problem.
This ensures that the test waits for the completion of the delay animation before proceeding with the next operation, thereby avoiding the problem.

It is advisable to utilize the [`waits`](https://www.selenium.dev/documentation/webdriver/waits/) methods inherent to Selenium as a substitute for `Thread.sleep()`.
It is advisable to utilize the [`waits`](https://www.selenium.dev/documentation/webdriver/waits/) methods inherent to Selenium as a substitute for `Thread.sleep()`.
:::

0 comments on commit b7ff5f4

Please sign in to comment.