Skip to content

PD panics when list resource-group with some resource group defined. #7206

Closed
@AndreMouche

Description

Bug Report

What did you do?

create some resource group
and try to list them.

What did you expect to see?

No panic and get all resource groups

What did you see instead?

PD panic

panic: json: unsupported value: NaN

goroutine 331601 [running]:
github.com/tikv/pd/pkg/mcs/resource_manager/server.(*ResourceGroup).Copy(0x40179af5d8?)
    /mnt/data1/jenkins/workspace/build-common@2/go/src/github.com/pingcap/pd/pkg/mcs/resource_manager/server/resource_group.go:68 +0x13c
github.com/tikv/pd/pkg/mcs/resource_manager/server.(*Manager).GetResourceGroupList(0x4000511ec0)
    /mnt/data1/jenkins/workspace/build-common@2/go/src/github.com/pingcap/pd/pkg/mcs/resource_manager/server/manager.go:245 +0x124
github.com/tikv/pd/pkg/mcs/resource_manager/server.(*Service).ListResourceGroups(0x4000209638?, {0x402a3abc80?, 0x3?}, 0x3?)
    /mnt/data1/jenkins/workspace/build-common@2/go/src/github.com/pingcap/pd/pkg/mcs/resource_manager/server/grpc_service.go:114 +0x74
github.com/pingcap/kvproto/pkg/resource_manager._ResourceManager_ListResourceGroups_Handler.func1({0x3891a90, 0x402a3abb30}, {0x2a33aa0?, 0x4035da1b80})
    /root/go/pkg/mod/github.com/pingcap/kvproto@v0.0.0-20230407040905-68d0eebd564a/pkg/resource_manager/resource_manager.pb.go:1868 +0x74
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1({0x3891a90?, 0x402a3abb30?}, {0x2a33aa0?, 0x4035da1b80?})
    /root/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:31 +0x9c
github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1({0x3891a90, 0x402a3abb30}, {0x2a33aa0, 0x4035da1b80}, 0x20?, 0x401cdd8e10)
    /root/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107 +0x74
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1({0x3891a90?, 0x402a3abb30?}, {0x2a33aa0?, 0x4035da1b80?})
    /root/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:34 +0x74
go.etcd.io/etcd/etcdserver/api/v3rpc.newUnaryInterceptor.func1({0x3891a90, 0x402a3abb30}, {0x2a33aa0?, 0x4035da1b80}, 0x3366471900000000?, 0x401cdd8e10)
    /root/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20220915004622-85b640cee793/etcdserver/api/v3rpc/interceptor.go:70 +0x2c4
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1({0x3891a90?, 0x402a3abb30?}, {0x2a33aa0?, 0x4035da1b80?})
    /root/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:34 +0x74
go.etcd.io/etcd/etcdserver/api/v3rpc.newLogUnaryInterceptor.func1({0x3891a90, 0x402a3abb30}, {0x2a33aa0, 0x4035da1b80}, 0x4035da1ba0, 0x401cdd8e10)
    /root/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20220915004622-85b640cee793/etcdserver/api/v3rpc/interceptor.go:77 +0x80
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1({0x3891a90, 0x402a3abb30}, {0x2a33aa0, 0x4035da1b80}, 0x4035da1ba0, 0x40281687c8)
    /root/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.0.1-0.20190118093823-f849b5445de4/chain.go:39 +0x17c
github.com/pingcap/kvproto/pkg/resource_manager._ResourceManager_ListResourceGroups_Handler({0x29e8040?, 0x4000209638}, {0x3891a90, 0x402a3abb30}, 0x4035caaae0, 0x400157c1b0)
    /root/go/pkg/mod/github.com/pingcap/kvproto@v0.0.0-20230407040905-68d0eebd564a/pkg/resource_manager/resource_manager.pb.go:1870 +0x12c
google.golang.org/grpc.(*Server).processUnaryRPC(0x4001cf2480, {0x38a10e0, 0x402dc29800}, 0x400cd0c300, 0x400243f590, 0x48e45c0, 0x0)
    /root/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1024 +0xb18
google.golang.org/grpc.(*Server).handleStream(0x4001cf2480, {0x38a10e0, 0x402dc29800}, 0x400cd0c300, 0x0)
    /root/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1313 +0x854
google.golang.org/grpc.(*Server).serveStreams.func1.1()
    /root/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:722 +0x84
created by google.golang.org/grpc.(*Server).serveStreams.func1
    /root/go/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:720 +0xdc
Stream closed EOF for prod-tidb/prod-redash-pd-0 (pd)

Deleting all resource groups stops the panics.

What version of PD are you using (pd-server -V)?

v7.1.0

Metadata

Assignees

Labels

affects-7.1This bug affects the 7.1.x(LTS) versions.affects-7.5This bug affects the 7.5.x(LTS) versions.report/customerCustomers have encountered this bug.severity/majortype/bugThe issue is confirmed as a bug.

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions