Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object Storage (LEP 20230430) #649

Closed
wants to merge 45 commits into from

Conversation

m-ildefons
Copy link
Contributor

User interface for managing the object stores of Longhorn.
This adds a new tab to the Longhorn UI, which is dedicated to managing object stores. It can list, create and delete object stores as well as show their operational status.
Each object store corresponds to an ObjectStore CRD in the K8s cluster, the Longhorn Manager implements a corresponding controller which orchestrates an s3gw instance to implement that object store.

Related: longhorn/longhorn-manager#2136
Related: longhorn/longhorn#6640

LEP: longhorn/longhorn#5832

Add an "Object Storage" tab for managing object endpoints.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
src/models/objectStore.js Show resolved Hide resolved
@votdev votdev enabled auto-merge October 26, 2023 09:50
@votdev votdev disabled auto-merge October 26, 2023 09:50
@longhorn longhorn deleted a comment from mergify bot Nov 2, 2023
@votdev votdev force-pushed the wip/object-endpoint branch 2 times, most recently from f98925b to 45ad8cf Compare November 2, 2023 09:35
votdev
votdev previously approved these changes Nov 2, 2023
votdev and others added 14 commits November 6, 2023 16:09
Signed-off-by: Volker Theile <vtheile@suse.com>
- Reload list after creating a new ObjectStore
- Add new `Administrate` menu which will redirect to the s3gw-ui
- Disable action menus if necessary

Signed-off-by: Volker Theile <vtheile@suse.com>
Signed-off-by: Volker Theile <vtheile@suse.com>
Signed-off-by: Volker Theile <vtheile@suse.com>
…eleted

Signed-off-by: Volker Theile <vtheile@suse.com>
Signed-off-by: Volker Theile <vtheile@suse.com>
Signed-off-by: Volker Theile <vtheile@suse.com>
Serve Longhorn UI from /longhorn sub path. This avoids conflicts with
s3gw instances and allows redirecting the browser to the S3 management
UI

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
- Fix model function `get` --> `put`
- Fix divider in action menu not having a key and thus throwing a
  warning

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Add endpoint creation to the object store creation dialogue

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Signed-off-by: Volker Theile <vtheile@suse.com>
Serve Longhorn UI from /longhorn sub path. This avoids conflicts with
s3gw instances and allows redirecting the browser to the S3 management
UI

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Make the endpoint input component work correctly by letting it propagate
the user input to the parent component

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
- Add sorter
- Init pagination correctly
- Cleanup store handling in objectStorage

Signed-off-by: Volker Theile <vtheile@suse.com>
Proxy object store UI in the nginx of the Longhorn UI. This allows for
port-forwarding of the Longhorn UI to also access the administrative
interface of the s3gw UI instances

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Copy link

mergify bot commented Nov 6, 2023

This pull request is now in conflicts. Could you fix it @m-ildefons? 🙏

Disable request body size limit for the object store UI location in the
nginx config. This allows uploading large files through the s3gw UI to
the object store.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Fix websocket connections in the object store page:
  - Name secrets, settings and objectstores in the data dependency model
  - Fix secrets model by adding state update functions for the websocket
  - Fix footer component
  - Ensure all resource names, types, namespaces etc. match what is
    exposed by the websocket API of the longhorn manager

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Add action menu entry and plumbing for stopping/restarting an object
store. This is useful to allow for certain kinds of volume maintenance
operations, since it stops the workload on the volume backing the object
store.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Display size and free space of the volume associated with an object
store in the object store view. This makes it easy for users to view the
amount of free space in an object store.

Fix filters in object store view: Allow object stores to be filtered by
their state and endpoints in addition to their name. This makes it easy
for users to find stopped object stores if they have too many to display
on a single page. It also makes it easy to find the object store
associated with a domain name, which otherwise would be hard if the name
of an object store has no relation to the domain name in use.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Fix flickering access/secret keys in create object store dialogue

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Fix volume expansion:
- don't allow a user to edit the size of an object store and make it
  smaller
- propagate the size from the edit menu to the backend, so the volume
  can be expanded

Fix from backup:
- add an input field to allow populating a new object store with data
  from a previous backup

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
remove superfluous getObjectStore function and code path. It is not
needed as there is no detailed view of object stores and the list view
contains all available information

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@PhanLe1010
Copy link

I couldn't figureout why we cannot access the s3gw-ui using your latest version at https://github.com/m-ildefons/s3gw-ui/tree/wip/longhorn-integration. Can you help?

Screenshot from 2023-11-20 17-03-18

@m-ildefons
Copy link
Contributor Author

This is odd. I've never seen that error, maybe @votdev has an idea where it comes from. Can you provide a reproducer or some kind of hint what your configuration looks like?

@PhanLe1010
Copy link

Hi @m-ildefons and @votdev The reproducing steps are:

  1. Deploy Longhorn with the images built from ObjectStore PRs
  2. The s3gw is quay.io/s3gw/s3gw:v0.23.0-rc3
  3. The s3gw UI image is built from this repo https://github.com/m-ildefons/s3gw-ui
  4. Accessing Longhorn UI through Rancher Proxy
    Screenshot from 2023-11-27 11-59-23
  5. Create an ObjectStore
  6. Try to access the ObjectStore UI by clicking Administrate button
    Screenshot from 2023-11-27 12-00-30
  7. See the error in the browser:
    Screenshot from 2023-11-27 12-03-03
  8. See the error log inside s3gw UI container:
    INFO:     10.42.123.238:40554 - "GET / HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40558 - "GET /styles.2dbe82fcf6fce2f5.css HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40560 - "GET /main.9326ceeba599d59e.js HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40562 - "GET /polyfills.b9a16c87a52c6b7a.js HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40564 - "GET /runtime.d615879041906b88.js HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40566 - "GET /lato-latin-400-normal.b7ffde2383bb16ba.woff2 HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40572 - "GET /styles.2dbe82fcf6fce2f5.css HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40574 - "GET /favicon.svg HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40576 - "GET /assets/i18n/%22en-us%22.json HTTP/1.1" 404 Not Found
    INFO:     10.42.123.238:40580 - "GET /assets/app-main.config.json HTTP/1.1" 200 OK
    2023-11-27T20:01:27.562799448Z INFO:     10.42.123.238:40578 - "GET /assets/app.config.json HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40582 - "GET /assets/i18n/%22en-us%22.json HTTP/1.1" 404 Not Found
    INFO:     10.42.123.238:40584 - "GET /lato-latin-400-normal.b7ffde2383bb16ba.woff2 HTTP/1.1" 200 OK
    INFO:     10.42.123.238:40588 - "GET /assets/i18n/%22en-us%22.json HTTP/1.1" 404 Not Found
    

Note that if we do kubectl port-forwarding to Longhorn UI instead of access it through Rancher proxy, we don't hit the issue

When deploying an object store, there can be situations where the json
object sent to the API is incorrectly constructed with null values. To
avoid this, construct valid json objects regardless of the setting for
the tls secret and domain name.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@votdev
Copy link
Contributor

votdev commented Nov 28, 2023

@innobead innobead closed this Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants