Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Object Storage (LEP 20230430) #649

Closed
wants to merge 45 commits into from
Closed

Object Storage (LEP 20230430) #649

wants to merge 45 commits into from

Conversation

m-ildefons
Copy link
Contributor

User interface for managing the object stores of Longhorn.
This adds a new tab to the Longhorn UI, which is dedicated to managing object stores. It can list, create and delete object stores as well as show their operational status.
Each object store corresponds to an ObjectStore CRD in the K8s cluster, the Longhorn Manager implements a corresponding controller which orchestrates an s3gw instance to implement that object store.

Related: longhorn/longhorn-manager#2136
Related: longhorn/longhorn#6640

LEP: longhorn/longhorn#5832

@m-ildefons
Add Object Storage Tab
54b4631 
Add an "Object Storage" tab for managing object endpoints.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
This was referenced Sep 6, 2023
Closed
Closed
@votdev votdev force-pushed the wip/object-endpoint branch 5 times, most recently from 1aef73b to e6195ba Compare September 13, 2023 07:47
This was referenced Sep 14, 2023
Closed
Closed
Closed
Closed
@m-ildefons m-ildefons changed the title [Draft] [DNM] Object Storage Object Storage (LEP 20230430) Sep 28, 2023
@m-ildefons m-ildefons marked this pull request as ready for review September 28, 2023 15:25
@m-ildefons m-ildefons requested a review from a team as a code owner September 28, 2023 15:25
votdev
votdev suggested changes Oct 4, 2023
View reviewed changes
src/models/objectStore.js Show resolved Hide resolved
votdev added a commit to jecluis/longhorn that referenced this pull request Oct 6, 2023
@votdev
Add UI related topics
ac2494e 
Referecens:
- longhorn/longhorn-ui#649
- https://github.com/longhorn/longhorn-manager/pull/2136/files#diff-cb740c237beef9090d8b3a15ca8b76aac83a3af2d52deadc382b4309916a715c
- https://github.com/longhorn/longhorn-manager/pull/2136/files#diff-c75a796f2c69c35ee5ab6293f23db5c8d5af9a241f4f2345e96b1fc41d72ddceR41

Signed-off-by: Volker Theile <vtheile@suse.com>
votdev added a commit to jecluis/longhorn that referenced this pull request Oct 12, 2023
@votdev
Add UI related topics
75de654 
Referecens:
- longhorn/longhorn-ui#649
- https://github.com/longhorn/longhorn-manager/pull/2136/files#diff-cb740c237beef9090d8b3a15ca8b76aac83a3af2d52deadc382b4309916a715c
- https://github.com/longhorn/longhorn-manager/pull/2136/files#diff-c75a796f2c69c35ee5ab6293f23db5c8d5af9a241f4f2345e96b1fc41d72ddceR41

Signed-off-by: Volker Theile <vtheile@suse.com>
@votdev votdev enabled auto-merge October 26, 2023 09:50
@votdev votdev disabled auto-merge October 26, 2023 09:50
@longhorn longhorn deleted a comment from mergify bot Nov 2, 2023
@votdev votdev force-pushed the wip/object-endpoint branch 2 times, most recently from f98925b to 45ad8cf Compare November 2, 2023 09:35
votdev
votdev previously approved these changes Nov 2, 2023
View reviewed changes
@votdev votdev self-requested a review November 2, 2023 10:56
votdev and others added 14 commits November 6, 2023 16:09
@votdev @m-ildefons
Populate edit dialog
961924d 
Signed-off-by: Volker Theile <vtheile@suse.com>
@votdev @m-ildefons
Various enhancements
825ed83 
- Reload list after creating a new ObjectStore
- Add new `Administrate` menu which will redirect to the s3gw-ui
- Disable action menus if necessary

Signed-off-by: Volker Theile <vtheile@suse.com>
@votdev @m-ildefons
Fix table sizing/resizing
9c7ce30 
Signed-off-by: Volker Theile <vtheile@suse.com>
@votdev @m-ildefons
Fix coloring of ObjectStore state and capitalize the text
543276d 
Signed-off-by: Volker Theile <vtheile@suse.com>
@votdev @m-ildefons
Disable Delete bulk action button if not all selected rows can be d…
25b9163 
…eleted

Signed-off-by: Volker Theile <vtheile@suse.com>
@votdev @m-ildefons
Fix operation column to the right side
504803c 
Signed-off-by: Volker Theile <vtheile@suse.com>
@votdev @m-ildefons
Add divider to menu
fdbbadb 
Signed-off-by: Volker Theile <vtheile@suse.com>
@m-ildefons
object store: serve longhorn UI from /longhorn
1e7b1c7 
Serve Longhorn UI from /longhorn sub path. This avoids conflicts with
s3gw instances and allows redirecting the browser to the S3 management
UI

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons
object store: fix model functions
c5e1281 
- Fix model function `get` --> `put`
- Fix divider in action menu not having a key and thus throwing a
  warning

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons
object store: endpoint creation
0f9ee0c 
Add endpoint creation to the object store creation dialogue

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
Signed-off-by: Volker Theile <vtheile@suse.com>
@m-ildefons
object store: serve longhorn UI from /longhorn
c03f4b7 
Serve Longhorn UI from /longhorn sub path. This avoids conflicts with
s3gw instances and allows redirecting the browser to the S3 management
UI

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons
object store: make endpoint input work
c766877 
Make the endpoint input component work correctly by letting it propagate
the user input to the parent component

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@votdev @m-ildefons
Various UI improvements
c676c4e 
- Add sorter
- Init pagination correctly
- Cleanup store handling in objectStorage

Signed-off-by: Volker Theile <vtheile@suse.com>
@m-ildefons
object store: proxy object store UI
2b94c3e 
Proxy object store UI in the nginx of the Longhorn UI. This allows for
port-forwarding of the Longhorn UI to also access the administrative
interface of the s3gw UI instances

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@mergify Mergify
Copy link

mergify bot commented Nov 6, 2023

This pull request is now in conflicts. Could you fix it @m-ildefons? 🙏

@m-ildefons
object store: disable request body size limit
de3a8a7 
Disable request body size limit for the object store UI location in the
nginx config. This allows uploading large files through the s3gw UI to
the object store.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons
object store: fix websockets
a6a514f 
Fix websocket connections in the object store page:
  - Name secrets, settings and objectstores in the data dependency model
  - Fix secrets model by adding state update functions for the websocket
  - Fix footer component
  - Ensure all resource names, types, namespaces etc. match what is
    exposed by the websocket API of the longhorn manager

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons
object store: stop/start object stores
e3363aa 
Add action menu entry and plumbing for stopping/restarting an object
store. This is useful to allow for certain kinds of volume maintenance
operations, since it stops the workload on the volume backing the object
store.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons
object store: display space usage, filters
7d78aef 
Display size and free space of the volume associated with an object
store in the object store view. This makes it easy for users to view the
amount of free space in an object store.

Fix filters in object store view: Allow object stores to be filtered by
their state and endpoints in addition to their name. This makes it easy
for users to find stopped object stores if they have too many to display
on a single page. It also makes it easy to find the object store
associated with a domain name, which otherwise would be hard if the name
of an object store has no relation to the domain name in use.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons
object store: fix flickering keys
29f1b7a 
Fix flickering access/secret keys in create object store dialogue

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons
object store: fix volume expansion, from backup
82d47cf 
Fix volume expansion:
- don't allow a user to edit the size of an object store and make it
  smaller
- propagate the size from the edit menu to the backend, so the volume
  can be expanded

Fix from backup:
- add an input field to allow populating a new object store with data
  from a previous backup

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@m-ildefons
object store: remove unused code path
8abb391 
remove superfluous getObjectStore function and code path. It is not
needed as there is no detailed view of object stores and the list view
contains all available information

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@PhanLe1010
Copy link

I couldn't figureout why we cannot access the s3gw-ui using your latest version at https://github.com/m-ildefons/s3gw-ui/tree/wip/longhorn-integration. Can you help?

Screenshot from 2023-11-20 17-03-18

@m-ildefons m-ildefons mentioned this pull request Nov 22, 2023
Closed
@m-ildefons
Copy link
Contributor Author

This is odd. I've never seen that error, maybe @votdev has an idea where it comes from. Can you provide a reproducer or some kind of hint what your configuration looks like?

@PhanLe1010
Copy link

Hi @m-ildefons and @votdev The reproducing steps are:

  1. Deploy Longhorn with the images built from ObjectStore PRs
  2. The s3gw is quay.io/s3gw/s3gw:v0.23.0-rc3
  3. The s3gw UI image is built from this repo https://github.com/m-ildefons/s3gw-ui
  4. Accessing Longhorn UI through Rancher Proxy
    Screenshot from 2023-11-27 11-59-23
  5. Create an ObjectStore
  6. Try to access the ObjectStore UI by clicking Administrate button
    Screenshot from 2023-11-27 12-00-30
  7. See the error in the browser:
    Screenshot from 2023-11-27 12-03-03
  8. See the error log inside s3gw UI container: 
    INFO:     10.42.123.238:40554 - "GET / HTTP/1.1" 200 OK
INFO:     10.42.123.238:40558 - "GET /styles.2dbe82fcf6fce2f5.css HTTP/1.1" 200 OK
INFO:     10.42.123.238:40560 - "GET /main.9326ceeba599d59e.js HTTP/1.1" 200 OK
INFO:     10.42.123.238:40562 - "GET /polyfills.b9a16c87a52c6b7a.js HTTP/1.1" 200 OK
INFO:     10.42.123.238:40564 - "GET /runtime.d615879041906b88.js HTTP/1.1" 200 OK
INFO:     10.42.123.238:40566 - "GET /lato-latin-400-normal.b7ffde2383bb16ba.woff2 HTTP/1.1" 200 OK
INFO:     10.42.123.238:40572 - "GET /styles.2dbe82fcf6fce2f5.css HTTP/1.1" 200 OK
INFO:     10.42.123.238:40574 - "GET /favicon.svg HTTP/1.1" 200 OK
INFO:     10.42.123.238:40576 - "GET /assets/i18n/%22en-us%22.json HTTP/1.1" 404 Not Found
INFO:     10.42.123.238:40580 - "GET /assets/app-main.config.json HTTP/1.1" 200 OK
2023-11-27T20:01:27.562799448Z INFO:     10.42.123.238:40578 - "GET /assets/app.config.json HTTP/1.1" 200 OK
INFO:     10.42.123.238:40582 - "GET /assets/i18n/%22en-us%22.json HTTP/1.1" 404 Not Found
INFO:     10.42.123.238:40584 - "GET /lato-latin-400-normal.b7ffde2383bb16ba.woff2 HTTP/1.1" 200 OK
INFO:     10.42.123.238:40588 - "GET /assets/i18n/%22en-us%22.json HTTP/1.1" 404 Not Found

Note that if we do kubectl port-forwarding to Longhorn UI instead of access it through Rancher proxy, we don't hit the issue

@m-ildefons
object store: fix TLS secret bug
4ce6b89 
When deploying an object store, there can be situations where the json
object sent to the API is incorrectly constructed with null values. To
avoid this, construct valid json objects regardless of the setting for
the tls secret and domain name.

Signed-off-by: Moritz Röhrich <moritz.rohrich@suse.com>
@votdev
Copy link
Contributor

votdev commented Nov 28, 2023

Screenshot from 2023-11-27 12-03-03

See https://github.com/aquarist-labs/s3gw/issues/846

@innobead innobead closed this Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@votdev votdev Awaiting requested review from votdev

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@m-ildefons @PhanLe1010 @votdev @innobead @smallteeths @mantissahz @yardenshoham