Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bad pointer in Go heap #373

Closed
trim21 opened this issue Jun 21, 2022 · 1 comment
Closed

bad pointer in Go heap #373

trim21 opened this issue Jun 21, 2022 · 1 comment

Comments

@trim21
Copy link
Contributor

trim21 commented Jun 21, 2022

I'm using latest go-json version 0.9.7, in my long-time-running server and I found it restart with some memory problem many time.

after digging, I find the reason it goccy/go-json.

I can provide a maybe re-production example (I can't always run into the memory error, You may need to run many times):

package main

import (
	"errors"
	"fmt"
	"io"
	"net/http"
	"runtime"
	"strconv"
	"sync"

	"github.com/goccy/go-json"
)

func main() {
	runtime.GOMAXPROCS(4)
	var err error = ErrNotFound
	var w io.Writer = io.Discard

	for j := 0; j < 100; j++ {
		var wg sync.WaitGroup
		wg.Add(100)
		for i := 0; i < 100; i++ {
			go func() {
				defer wg.Done()

				var e HTTPError
				if !errors.As(err, &e) {
					return
				}
				// handle expected http error
				data := Error{
					Title:       http.StatusText(e.Code),
					Description: e.Msg,
					Details: D{
						Error:       err.Error(),
						Path:        "ppa",
						Method:      "vv",
						QueryString: "ww",
					},
				}
				b, err := json.MarshalNoEscape(data)
				if err != nil {
					fmt.Println("unexpected error", err)
					panic(err)
				}

				w.Write(b)
			}()

		}
		wg.Wait()
	}

}

var ErrNotFound = NewError(http.StatusNotFound, "resource can't be found in the database or has been removed")

// Error default error response.
type Error struct {
	Title       string      `json:"title"`
	Details     interface{} `json:"details,omitempty"`
	Description string      `json:"description"`
}

var _ error = HTTPError{}

type HTTPError struct {
	Msg  string
	Code int
}

func NewError(code int, message string) error {
	return HTTPError{Code: code, Msg: message}
}

func (e HTTPError) Error() string {
	return strconv.Itoa(e.Code) + ": " + e.Msg
}

type D struct {
	Error       string `json:"error,omitempty"`
	Path        string `json:"path,omitempty"`
	Method      string `json:"method,omitempty"`
	QueryString string `json:"query_string,omitempty"`
}

run it with go run -race main.go, you may see many kind of errors

  1. DATA RACE
==================
WARNING: DATA RACE
Read at 0x00c0004dc6c0 by goroutine 89:
  github.com/goccy/go-json/internal/encoder.appendNormalizedHTMLString()
      C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/string.go:49 +0x28e
  github.com/goccy/go-json/internal/encoder.AppendString()
      C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/string.go:28 +0xdb
  github.com/goccy/go-json/internal/encoder/vm.Run()
      C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/vm/vm.go:3631 +0x3a2b0
  github.com/goccy/go-json.encodeRunCode()
      C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/encode.go:310 +0x271
  github.com/goccy/go-json.encodeNoEscape()
      C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/encode.go:261 +0x36c
  github.com/goccy/go-json.marshalNoEscape()
      C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/encode.go:174 +0xe4
  github.com/goccy/go-json.MarshalNoEscape()
      C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/json.go:176 +0x344
  main.main.func1()
      C:/Users/Trim21/proj/app/main.go:42 +0x2fc
  1. bad ptr with reflect
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x3f pc=0x9e0739]

goroutine 6438 [running]:
reflect.(*rtype).Kind(...)
        C:/Users/Trim21/.g/go/src/reflect/type.go:823
reflect.implements(0xbf9fa0, 0x3f)
        C:/Users/Trim21/.g/go/src/reflect/type.go:1548 +0xb9
reflect.(*rtype).Implements(0x3f?, {0xcda640, 0xbf9fa0?})
        C:/Users/Trim21/.g/go/src/reflect/type.go:1503 +0x79
github.com/goccy/go-json/internal/runtime.(*Type).Implements(...)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/runtime/rtype.go:96
github.com/goccy/go-json/internal/encoder.(*Compiler).implementsMarshalJSONType(0x0?, 0x0?)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/compiler.go:869 +0x4b
github.com/goccy/go-json/internal/encoder.(*Compiler).implementsMarshalJSON(0x0?, 0x0?)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/compiler.go:826 +0x3b
github.com/goccy/go-json/internal/encoder.(*Compiler).typeToCode(0x0?, 0x3f)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/compiler.go:142 +0x5b
github.com/goccy/go-json/internal/encoder.(*Compiler).compile(0xbf71e0?, 0x3f)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/compiler.go:100 +0x59
github.com/goccy/go-json/internal/encoder.compileToGetCodeSetSlowPath(0x76?)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/compiler.go:58 +0x125
github.com/goccy/go-json/internal/encoder.CompileToGetCodeSet(0xbef8a0?, 0x3f)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/compiler_race.go:14 +0xd2
github.com/goccy/go-json/internal/encoder/vm.Run(0xc00049f1e0, {0xc0004c7800, 0x0, 0x400}, 0xc0000aa000)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/vm/vm.go:202 +0x5aa65
github.com/goccy/go-json.encodeRunCode(0xc00049f1e0, {0xc0004c7800, 0x0, 0x400}, 0xc0003fc9c0?)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/encode.go:310 +0x272
github.com/goccy/go-json.encodeNoEscape(0xc00049f1e0, {0xc07000, 0xc000493f20})
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/encode.go:261 +0x36d
github.com/goccy/go-json.marshalNoEscape({0xc07000, 0xc0001f7f20})
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/encode.go:174 +0xe5
github.com/goccy/go-json.MarshalNoEscape(...)
        C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/json.go:176
main.main.func1()
        C:/Users/Trim21/proj/app/main.go:42 +0x345
created by main.main
        C:/Users/Trim21/proj/app/main.go:24 +0xf4
exit status 2
  1. bad ptr with gc
runtime: pointer 0xc0004a3f30 to unallocated span span.base()=0xc000480000 span.limit=0xc000490000 span.state=0
fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)

goroutine 2935 [running]:
runtime.throw({0x9e3834?, 0xc000474020?})
	C:/Users/Trim21/.g/go/src/runtime/panic.go:992 +0x76 fp=0xc00050f990 sp=0xc00050f960 pc=0x733196
runtime.badPointer(0x1b6e7b550d0, 0x0?, 0x0, 0x0?)
	C:/Users/Trim21/.g/go/src/runtime/mbitmap.go:368 +0x150 fp=0xc00050f9e0 sp=0xc00050f990 pc=0x710f90
runtime.findObject(0x0?, 0x988700?, 0xc0004ebc15?)
	C:/Users/Trim21/.g/go/src/runtime/mbitmap.go:410 +0xb6 fp=0xc00050fa18 sp=0xc00050f9e0 pc=0x711136
runtime.checkptrBase(0xc0004a3f30)
	C:/Users/Trim21/.g/go/src/runtime/checkptr.go:94 +0x45 fp=0xc00050fa40 sp=0xc00050fa18 pc=0x703ba5
runtime.checkptrStraddles(0xc000474028?, 0x0?)
	C:/Users/Trim21/.g/go/src/runtime/checkptr.go:47 +0x4e fp=0xc00050fa68 sp=0xc00050fa40 pc=0x703a0e
runtime.checkptrAlignment(0x99f8a0?, 0x0?, 0xc0000121a6?)
	C:/Users/Trim21/.g/go/src/runtime/checkptr.go:25 +0x4d fp=0xc00050fa88 sp=0xc00050fa68 pc=0x70394d
github.com/goccy/go-json/internal/encoder/vm.Run(0xc00044fc70, {0xc0004ebc00, 0x0, 0x400}, 0xc000074000)
	C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/internal/encoder/vm/vm.go:192 +0x5a7dd fp=0xc000517c28 sp=0xc00050fa88 pc=0x9888dd
github.com/goccy/go-json.encodeRunCode(0xc00044fc70, {0xc0004ebc00, 0x0, 0x400}, 0xc0002cfe60?)
	C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/encode.go:310 +0x272 fp=0xc000517ca8 sp=0xc000517c28 pc=0x98b972
github.com/goccy/go-json.encodeNoEscape(0xc00044fc70, {0x9b7000, 0xc000517f20})
	C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/encode.go:261 +0x36d fp=0xc000517d68 sp=0xc000517ca8 pc=0x98b5cd
github.com/goccy/go-json.marshalNoEscape({0x9b7000, 0xc0004a3f20})
	C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/encode.go:174 +0xe5 fp=0xc000517de8 sp=0xc000517d68 pc=0x98ab45
github.com/goccy/go-json.MarshalNoEscape(...)
	C:/Users/Trim21/go/pkg/mod/github.com/goccy/go-json@v0.9.7/json.go:176
main.main.func1()
	C:/Users/Trim21/proj/app/main.go:42 +0x345 fp=0xc000517fe0 sp=0xc000517de8 pc=0x98c665
runtime.goexit()
	C:/Users/Trim21/.g/go/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc000517fe8 sp=0xc000517fe0 pc=0x7610a1
created by main.main
	C:/Users/Trim21/proj/app/main.go:24 +0xf4
@trim21 trim21 closed this as completed Jun 21, 2022
@trim21
Copy link
Contributor Author

trim21 commented Jun 21, 2022 •
edited
Loading

Sorry, I found that I should not use MarshalNoEscape in this case. I thought it's "no html escape"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@trim21