feat: service restart provides now per node status by harshavardhana · Pull Request #4827 · minio/mc

harshavardhana · 2024-01-19T05:33:46Z

Community Contribution License

All community contributions in this pull request are licensed to the project maintainers
under the terms of the Apache 2 license.
By creating this pull request I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 license.

Description

Motivation and Context

How to test this PR?

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Optimization (provides speedup with no functional changes)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

Fixes a regression (If yes, please add commit-id or PR # here)
Unit tests added/updated
Internal documentation updated
Create a documentation update request here

harshavardhana · 2024-01-19T05:34:10Z

This PR is not ready until minio/madmin-go#263 is merged

klauspost · 2024-01-19T11:43:17Z

Server PR: minio/minio#18826 for reference.

harshavardhana · 2024-01-26T04:43:15Z

This PR is ready with all the changes both restart/update so it is good to be reviewed and merged right away.

shtripat · 2024-01-26T15:13:03Z

In a local single node single disk setup started as minio server ./test the output for mc admin service restart ALIAS shows as below

./mc admin service restart myminio 
Restart command successfully sent to `myminio`. Type Ctrl-C to quit or wait to follow the status of the restart process.
┌────────────────┬────────┐
│ HOST           │ STATUS │
├────────────────┼────────┤
│ 127.0.0.1:9000 │ ✔      │
│                │ ✔      │
└────────────────┴────────┘
...
Restarted `myminio` successfully in 1 seconds

The second tick mark seems unwanted for this scenario.

whereas for a multi node setup looks fine as below

./mc admin service restart myminio
Restart command successfully sent to `myminio`. Type Ctrl-C to quit or wait to follow the status of the restart process.
┌────────────────┬────────┐
│ HOST           │ STATUS │
├────────────────┼────────┤
│ 127.0.0.1:9000 │ ✔      │
│ 127.0.0.1:9001 │ ✔      │
│ 127.0.0.1:9002 │ ✔      │
│ 127.0.0.1:9003 │ ✔      │
└────────────────┴────────┘

...!!
Restarted `myminio` successfully in 3 seconds

Feature wise code looks good to me.

harshavardhana · 2024-01-26T16:43:36Z

Gonna fix the single node output.

shtripat · 2024-01-26T17:13:26Z

Gonna fix the single node output.

Also with SNSD mode ./mc admin update myminio "https://dl.min.io/server/minio/release/linux-amd64/minio.RELEASE.2024-01-18T22-51-28Z.sha256sum" failed with error on MinIO side as below

panic: "POST /minio/admin/v3/update?dry-run=false&type=2&updateURL=https%3A%2F%2Fdl.min.io%2Fserver%2Fminio%2Frelease%2Flinux-amd64%2Fminio.RELEASE.2024-01-18T22-51-28Z.sha256sum": runtime error: invalid memory address or nil pointer dereference
goroutine 738 [running]:
runtime/debug.Stack()
	runtime/debug/stack.go:24 +0x5e
github.com/minio/minio/cmd.serverMain.func8.setCriticalErrorHandler.func2.1()
	github.com/minio/minio/cmd/generic-handlers.go:555 +0x97
panic({0x28080e0?, 0x62c2cf0?})
	runtime/panic.go:914 +0x21f
github.com/minio/pkg/v2/workers.(*Workers).Wait(...)
	github.com/minio/pkg/v2@v2.0.8/workers/workers.go:62
github.com/minio/minio/cmd.(*NotificationGroup).Wait(...)
	github.com/minio/minio/cmd/notification.go:101
github.com/minio/minio/cmd.(*NotificationSys).VerifyBinary(0xc001317740, {0x4c51ec0?, 0xc000ecd860}, 0xc08ff0e240, {0xc08ff01e40, 0x20, 0x40}, {0xc08ff195b1, 0x22}, {0xc09c042000, ...})
	github.com/minio/minio/cmd/notification.go:390 +0x38d
github.com/minio/minio/cmd.adminAPIHandlers.ServerUpdateV2Handler({}, {0x4c4c5e0, 0xc000cd2280}, 0xc08ff14300)
	github.com/minio/minio/cmd/admin-handlers.go:148 +0x645
net/http.HandlerFunc.ServeHTTP(0x52f303b?, {0x4c4c5e0?, 0xc000cd2280?}, 0x4c7f108?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.adminMiddleware.func1.httpTraceAll.httpTrace.func1({0x4c4c5e0, 0xc000cd2280}, 0xc000cd2280?)
	github.com/minio/minio/cmd/http-tracer.go:189 +0xef
github.com/minio/minio/cmd.adminMiddleware.func1({0x4c4c5e0, 0xc000cd2280}, 0xc08ff14200)
	github.com/minio/minio/cmd/admin-router.go:132 +0x2df
net/http.HandlerFunc.ServeHTTP(0x62d40a0?, {0x4c4c5e0?, 0xc000cd2280?}, 0x4?)
	net/http/server.go:2136 +0x29
github.com/klauspost/compress/gzhttp.NewWrapper.func1.1({0x4c4bc50, 0xc08ff30000}, 0x1b?)
	github.com/klauspost/compress@v1.17.4/gzhttp/compress.go:497 +0x37d
net/http.HandlerFunc.ServeHTTP(0xc000ecd1a0?, {0x4c4bc50?, 0xc08ff30000?}, 0x0?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setBucketForwardingMiddleware.func1({0x4c4bc50, 0xc08ff30000}, 0xc08ff14200)
	github.com/minio/minio/cmd/generic-handlers.go:460 +0x2fc
net/http.HandlerFunc.ServeHTTP(0x7f193e5eeee0?, {0x4c4bc50?, 0xc08ff30000?}, 0xc000100000?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setUploadForwardingMiddleware.func1({0x4c4bc50, 0xc08ff30000}, 0xc08ff14200)
	github.com/minio/minio/cmd/generic-handlers.go:573 +0x1e5
net/http.HandlerFunc.ServeHTTP(0x2b3ee40?, {0x4c4bc50?, 0xc08ff30000?}, 0x39?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setRequestValidityMiddleware.func1({0x4c4bc50?, 0xc08ff30000}, 0xc08ff14200)
	github.com/minio/minio/cmd/generic-handlers.go:443 +0x168d
net/http.HandlerFunc.ServeHTTP(0xc000ecd500?, {0x4c4bc50?, 0xc08ff30000?}, 0xc000fe4cf0?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setRequestLimitMiddleware.func1({0x4c4bc50?, 0xc08ff30000}, 0xc08ff14200)
	github.com/minio/minio/cmd/generic-handlers.go:134 +0x662
net/http.HandlerFunc.ServeHTTP(0x2f?, {0x4c4bc50?, 0xc08ff30000?}, 0x2253fab?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setCrossDomainPolicyMiddleware.func1({0x4c4bc50?, 0xc08ff30000?}, 0x6?)
	github.com/minio/minio/cmd/crossdomain-xml-handler.go:42 +0xf1
net/http.HandlerFunc.ServeHTTP(0xc08ff14200?, {0x4c4bc50?, 0xc08ff30000?}, 0xc0017e2110?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setBrowserRedirectMiddleware.func1({0x4c4bc50, 0xc08ff30000}, 0xc08ff14200)
	github.com/minio/minio/cmd/generic-handlers.go:164 +0x19c
net/http.HandlerFunc.ServeHTTP(0x0?, {0x4c4bc50?, 0xc08ff30000?}, 0x871c46f?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.setAuthMiddleware.func1({0x4c4bc50, 0xc08ff30000}, 0xc08ff14200)
	github.com/minio/minio/cmd/auth-handler.go:636 +0x6a2
net/http.HandlerFunc.ServeHTTP(0x4c51ec0?, {0x4c4bc50?, 0xc08ff30000?}, 0x4c2a490?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.httpTracerMiddleware.func1({0x4c4c1f0, 0xc08ff1a000}, 0xc08ff14100)
	github.com/minio/minio/cmd/http-tracer.go:89 +0x3b4
net/http.HandlerFunc.ServeHTTP(0x29faa00?, {0x4c4c1f0?, 0xc08ff1a000?}, 0xa?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.addCustomHeadersMiddleware.func1({0x4c4c1f0, 0xc08ff1a000}, 0xc000ecd3e0?)
	github.com/minio/minio/cmd/generic-handlers.go:538 +0x438
net/http.HandlerFunc.ServeHTTP(0xc08ff14000?, {0x4c4c1f0?, 0xc08ff1a000?}, 0x69d73f?)
	net/http/server.go:2136 +0x29
github.com/minio/mux.(*Router).ServeHTTP(0xc000dcaa80, {0x4c4c1f0, 0xc08ff1a000}, 0xc08ff14000)
	github.com/minio/mux@v1.9.0/mux.go:214 +0x1f7
github.com/minio/minio/cmd.corsHandler.(*Cors).Handler.func2({0x4c4c1f0, 0xc08ff1a000}, 0xc08ff14000)
	github.com/rs/cors@v1.10.1/cors.go:281 +0x184
net/http.HandlerFunc.ServeHTTP(0x4186e8?, {0x4c4c1f0?, 0xc08ff1a000?}, 0xf?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/cmd.serverMain.func8.setCriticalErrorHandler.func2({0x4c4c1f0?, 0xc08ff1a000?}, 0x63474e0?)
	github.com/minio/minio/cmd/generic-handlers.go:562 +0x78
net/http.HandlerFunc.ServeHTTP(0xc001646ac8?, {0x4c4c1f0?, 0xc08ff1a000?}, 0x727385?)
	net/http/server.go:2136 +0x29
github.com/minio/minio/internal/http.(*Server).Init.func1({0x4c4c1f0?, 0xc08ff1a000?}, 0xc00131e300?)
	github.com/minio/minio/internal/http/server.go:123 +0x19c
net/http.HandlerFunc.ServeHTTP(0x4109a5?, {0x4c4c1f0?, 0xc08ff1a000?}, 0xc08ff1a001?)
	net/http/server.go:2136 +0x29
net/http.serverHandler.ServeHTTP({0x4c44f38?}, {0x4c4c1f0?, 0xc08ff1a000?}, 0x6?)
	net/http/server.go:2938 +0x8e
net/http.(*conn).serve(0xc08ff0e000, {0x4c51ec0, 0xc000e72570})
	net/http/server.go:2009 +0x5f4
created by net/http.(*Server).Serve in goroutine 94
	net/http/server.go:3086 +0x5cb

whereas multi node showed message

Server update request sent successfully `myminio`
┌────────────────┬────────────────────────────────────────────────────────────────────┐
│ HOST           │ STATUS                                                             │
├────────────────┼────────────────────────────────────────────────────────────────────┤
│ 127.0.0.1:9002 │ server is already running the latest version: 2023-08-21T19-40-07Z │
│ 127.0.0.1:9003 │ server is already running the latest version: 2023-08-21T19-40-07Z │
│ 127.0.0.1:9000 │ server is already running the latest version: 2023-08-21T19-40-07Z │
│ 127.0.0.1:9001 │ server is already running the latest version: 2023-08-21T19-40-07Z │
└────────────────┴────────────────────────────────────────────────────────────────────┘

harshavardhana · 2024-01-26T20:32:07Z

Yes server side is fixed @shtripat

commit c88308cf0e6401ca034561357b6e7c932b11a516 (HEAD -> master, origin/master)
Author: Harshavardhana <harsha@minio.io>
Date:   Fri Jan 26 12:07:03 2024 -0800

    avoid 'panic' on mc admin update for single drive setup (#18876)

harshavardhana force-pushed the per-node branch 2 times, most recently from bdd3f07 to 0478e4d Compare January 24, 2024 23:04

bh4t assigned harshavardhana Jan 24, 2024

harshavardhana force-pushed the per-node branch from 0478e4d to 453dba6 Compare January 25, 2024 23:09

harshavardhana mentioned this pull request Jan 25, 2024

add new update v2 that updates per node, allows idempotent behavior minio/minio#18859

Merged

8 tasks

harshavardhana force-pushed the per-node branch from 453dba6 to 85d6fee Compare January 26, 2024 00:12

feat: service restart provides now per node status

c13b1c8

harshavardhana requested review from jiuker, shtripat and vadmeste January 26, 2024 04:43

jiuker approved these changes Jan 26, 2024

View reviewed changes

klauspost approved these changes Jan 26, 2024

View reviewed changes

harshavardhana force-pushed the per-node branch from 63a3fcb to 76ce414 Compare January 26, 2024 19:31

feat: service update provides now per node status

e11167d

harshavardhana force-pushed the per-node branch from 76ce414 to e11167d Compare January 26, 2024 20:17

harshavardhana merged commit 9a5ef7c into minio:master Jan 26, 2024

harshavardhana deleted the per-node branch January 26, 2024 20:32

djwfyi mentioned this pull request Feb 5, 2024

[RELEASE] mc RELEASE.2024-01-28T16-23-14Z minio/docs#1120

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: service restart provides now per node status#4827

feat: service restart provides now per node status#4827
harshavardhana merged 2 commits into
minio:masterfrom
harshavardhana:per-node

harshavardhana commented Jan 19, 2024

Uh oh!

harshavardhana commented Jan 19, 2024

Uh oh!

klauspost commented Jan 19, 2024

Uh oh!

harshavardhana commented Jan 26, 2024

Uh oh!

shtripat commented Jan 26, 2024 •

edited

Loading

Uh oh!

harshavardhana commented Jan 26, 2024

Uh oh!

shtripat commented Jan 26, 2024

Uh oh!

harshavardhana commented Jan 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

harshavardhana commented Jan 19, 2024

Community Contribution License

Description

Motivation and Context

How to test this PR?

Types of changes

Checklist:

Uh oh!

harshavardhana commented Jan 19, 2024

Uh oh!

klauspost commented Jan 19, 2024

Uh oh!

harshavardhana commented Jan 26, 2024

Uh oh!

shtripat commented Jan 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

harshavardhana commented Jan 26, 2024

Uh oh!

shtripat commented Jan 26, 2024

Uh oh!

harshavardhana commented Jan 26, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

shtripat commented Jan 26, 2024 •

edited

Loading