Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data race when mutating underlying reader #479

Closed
dsnet opened this issue Jan 28, 2022 · 1 comment
Closed

Data race when mutating underlying reader #479

dsnet opened this issue Jan 28, 2022 · 1 comment

Comments

@dsnet
Copy link
Contributor

dsnet commented Jan 28, 2022

I expect the following code to work fine:

r := bytes.NewReader([]byte(""))
zr, _ := zstd.NewReader(r)
r.Seek(0, io.SeekStart)
zr.Reset(r)

However, running this snippet reports a data race:

==================
WARNING: DATA RACE
Read at 0x00c0000ac1c8 by goroutine 31:
  bytes.(*Reader).Read()
      go/src/bytes/reader.go:41 +0x58
  io.ReadAtLeast()
      go/src/io/io.go:328 +0xdd
  io.ReadFull()
      go/src/io/io.go:347 +0x75
  github.com/klauspost/compress/zstd.(*readerWrapper).readSmall()
      go/src/github.com/klauspost/compress/zstd/bytebuf.go:88 +0x2c
  github.com/klauspost/compress/zstd.(*frameDec).reset()
      go/src/github.com/klauspost/compress/zstd/framedec.go:83 +0xc6
  github.com/klauspost/compress/zstd.(*Decoder).startStreamDecoder()
      go/src/github.com/klauspost/compress/zstd/decoder.go:493 +0x464
  github.com/klauspost/compress/zstd.(*Decoder).Reset·dwrap·3()
      go/src/github.com/klauspost/compress/zstd/decoder.go:202 +0x47

Previous write at 0x00c0000ac1c8 by main goroutine:
  bytes.(*Reader).Seek()
      go/src/bytes/reader.go:133 +0xdb
  main.main()
      main.go:13 +0xb5

Goroutine 31 (running) created at:
  github.com/klauspost/compress/zstd.(*Decoder).Reset()
      go/src/github.com/klauspost/compress/zstd/decoder.go:202 +0x464
  github.com/klauspost/compress/zstd.NewReader()
      go/src/github.com/klauspost/compress/zstd/decoder.go:109 +0x728
  main.main()
      main.go:12 +0xaf
==================

It's a bit fishy that NewReader spawns go routines, but that's okay so long as the effect is not observable by the end user. However, in this situation, the effect is observable since the go routine continues to touch r after NewReader has returned. This causes a race with the call to r.Seek.

My current work-around is to call zr.Reset(nil) before I call r.Seek, but that seems like a hack.

Related #477

@klauspost
Copy link
Owner

@dsnet What you are proposing is IMO bad practice. You rely on a certain behavior of the decoder. What you are proposing will break, probably even for non-goroutine decoders - there will likely be buffers no matter what you do.

Prepare your source before sending it and don't mutate it before calling Close or Reset.

Yes, as part of #477 there will be a mode that doesn't spawn goroutines. That said, don't mutate the input, unless you are dealing with the consequences.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants