Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DataRace in IndexMergeReaderExec #31229

Closed
guo-shaoge opened this issue Dec 31, 2021 · 1 comment · Fixed by #31230
Closed

DataRace in IndexMergeReaderExec #31229

guo-shaoge opened this issue Dec 31, 2021 · 1 comment · Fixed by #31230
Assignees
@guo-shaoge
Labels
affects-5.4 This bug affects 5.4.x versions. severity/major type/bug The issue is confirmed as a bug.

Comments

@guo-shaoge
Copy link
Collaborator

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

cd $working_dir/tidb/planner/core && go test . -run TestT -check.f testIntegrationSuite.TestIndexMergeWithCorrelatedColumns -race -count 100

2. What did you expect to see? (Required)

runs ok.

3. What did you see instead (Required)

==================
WARNING: DATA RACE
Write at 0x00c005dc57e0 by goroutine 182:
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startPartialIndexWorker.func1.1()
      /go/tidb/executor/index_merge_reader.go:291 +0x444
  github.com/pingcap/tidb/util.WithRecovery()
      /go/tidb/util/misc.go:100 +0x77
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startPartialIndexWorker.func1()
      /go/tidb/executor/index_merge_reader.go:278 +0x290
Previous write at 0x00c005dc57e0 by goroutine 176:
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startPartialIndexWorker()
      /go/tidb/executor/index_merge_reader.go:359 +0x5a5
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startWorkers()
      /go/tidb/executor/index_merge_reader.go:214 +0x1cb
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).Next()
      /go/tidb/executor/index_merge_reader.go:629 +0x8e
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*StreamAggExec).consumeCurGroupRowsAndFetchChild()
      /go/tidb/executor/aggregate.go:1367 +0x13e
  github.com/pingcap/tidb/executor.(*StreamAggExec).consumeOneGroup()
      /go/tidb/executor/aggregate.go:1295 +0xee
  github.com/pingcap/tidb/executor.(*StreamAggExec).Next()
      /go/tidb/executor/aggregate.go:1284 +0x15d
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*NestedLoopApplyExec).fetchAllInners()
      /go/tidb/executor/join.go:979 +0x6bd
  github.com/pingcap/tidb/executor.(*NestedLoopApplyExec).Next()
      /go/tidb/executor/join.go:1032 +0x54d
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*projectionInputFetcher).run()
      /go/tidb/executor/projection.go:381 +0x468
  github.com/pingcap/tidb/executor.(*ProjectionExec).prepare·dwrap·115()
      /go/tidb/executor/projection.go:272 +0x58
Goroutine 182 (running) created at:
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startPartialIndexWorker()
      /go/tidb/executor/index_merge_reader.go:275 +0x598
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).startWorkers()
      /go/tidb/executor/index_merge_reader.go:214 +0x1cb
  github.com/pingcap/tidb/executor.(*IndexMergeReaderExecutor).Next()
      /go/tidb/executor/index_merge_reader.go:629 +0x8e
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*StreamAggExec).consumeCurGroupRowsAndFetchChild()
      /go/tidb/executor/aggregate.go:1367 +0x13e
  github.com/pingcap/tidb/executor.(*StreamAggExec).consumeOneGroup()
      /go/tidb/executor/aggregate.go:1295 +0xee
  github.com/pingcap/tidb/executor.(*StreamAggExec).Next()
      /go/tidb/executor/aggregate.go:1284 +0x15d
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*NestedLoopApplyExec).fetchAllInners()
      /go/tidb/executor/join.go:979 +0x6bd
  github.com/pingcap/tidb/executor.(*NestedLoopApplyExec).Next()
      /go/tidb/executor/join.go:1032 +0x54d
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*projectionInputFetcher).run()
      /go/tidb/executor/projection.go:381 +0x468
  github.com/pingcap/tidb/executor.(*ProjectionExec).prepare·dwrap·115()
      /go/tidb/executor/projection.go:272 +0x58
Goroutine 176 (running) created at:
  github.com/pingcap/tidb/executor.(*ProjectionExec).prepare()
      /go/tidb/executor/projection.go:272 +0xc08
  github.com/pingcap/tidb/executor.(*ProjectionExec).parallelExecute()
      /go/tidb/executor/projection.go:209 +0xca
  github.com/pingcap/tidb/executor.(*ProjectionExec).Next()
      /go/tidb/executor/projection.go:182 +0xc4
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*SortExec).fetchRowChunks()
      /go/tidb/executor/sort.go:195 +0xf73
  github.com/pingcap/tidb/executor.(*SortExec).Next()
      /go/tidb/executor/sort.go:113 +0xc4
  github.com/pingcap/tidb/executor.Next()
      /go/tidb/executor/executor.go:286 +0x58f
  github.com/pingcap/tidb/executor.(*recordSet).Next()
      /go/tidb/executor/adapter.go:149 +0x13b
  github.com/pingcap/tidb/session.(*execStmtResult).Next()
      <autogenerated>:1 +0x7b
  github.com/pingcap/tidb/session.GetRows4Test()
      /go/tidb/session/tidb.go:325 +0xea
  github.com/pingcap/tidb/session.ResultSetToStringSlice()
      /go/tidb/session/tidb.go:343 +0xb4
  github.com/pingcap/tidb/util/testkit.(*TestKit).ResultSetToResultWithCtx()
      /go/tidb/util/testkit/testkit.go:418 +0xbd
  github.com/pingcap/tidb/util/testkit.(*TestKit).ResultSetToResult()
      /go/tidb/util/testkit/testkit.go:413 +0x418
  github.com/pingcap/tidb/util/testkit.(*TestKit).MustQuery()
      /go/tidb/util/testkit/testkit.go:356 +0x420
  github.com/pingcap/tidb/planner/core_test.(*testIntegrationSuite).TestIndexMergeWithCorrelatedColumns()
      /go/tidb/planner/core/integration_test.go:5149 +0x4fb
  runtime.call16()
      /usr/local/go/src/runtime/asm_amd64.s:625 +0x48
  reflect.Value.Call()
      /usr/local/go/src/reflect/value.go:339 +0xd7
  github.com/pingcap/check.(*suiteRunner).forkTest.func1()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:850 +0xa54
  github.com/pingcap/check.(*suiteRunner).forkCall.func1()
      /home/prow/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739 +0x12b
==================

4. What is your TiDB version? (Required)

master
@guo-shaoge guo-shaoge added the type/bug The issue is confirmed as a bug. label Dec 31, 2021
@guo-shaoge guo-shaoge self-assigned this Dec 31, 2021
@guo-shaoge guo-shaoge mentioned this issue Dec 31, 2021
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this issue Dec 31, 2021
@guo-shaoge
executor: fix data race in IndexMergeReaderExec (#31230)
ceed548 
close #31229
@github-actions
Copy link

Please check whether the issue should be labeled with 'affects-x.y' or 'fixes-x.y.z', and then remove 'needs-more-info' label.

@ti-srebot ti-srebot mentioned this issue Dec 31, 2021
Merged
12 tasks
@djshow832 djshow832 added severity/major affects-5.4 This bug affects 5.4.x versions. labels Jan 4, 2022
rebelice pushed a commit to TiInterstellar/tidb that referenced this issue Jan 5, 2022
@crazycs520 @xhebox
@guo-shaoge @knull-cn @mengxin9014
hello world (#2)
5ab74c9 
* topsql: make topsql enable only be controlled by pub/sub sink (pingcap#31209)

* ddl: support batch create table  (pingcap#28763)

* executor: fix data race in IndexMergeReaderExec (pingcap#31230)

close pingcap#31229

* server: filter the EOF error for normal closed at handshake  (pingcap#31081)

close pingcap#31063

* expression: change date add function return type (pingcap#28133)

close pingcap#27573

* support create interval partition

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support create interval partition (support int/timestamp partition key)

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* parser: support alter table partitions move engine statement

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support ddl operation

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support interval partition manager

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support interval partition manager handle job framwork

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support auto create interval partition when insert meet no partition suitable error

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix cancel job and load old job then continue to do

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* make partition readonly work(not allow to insert/update/delete)

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add begin,end time in tables

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* tiny fix for auto create interval partition in concurrent case

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* init

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* init

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* todo: remove flag

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix dumpling

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove data in aws s3 when drop/truncate table/partition

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* make hello world work

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove debug info

Signed-off-by: crazycs520 <crazycs520@gmail.com>

Co-authored-by: xhe <xw897002528@gmail.com>
Co-authored-by: guo-shaoge <shaoge1994@163.com>
Co-authored-by: knull-cn <hu__haifeng@163.com>
Co-authored-by: Meng Xin <tregoldmeng@gmail.com>
ti-chi-bot pushed a commit that referenced this issue Jan 6, 2022
@ti-srebot
executor: fix data race in IndexMergeReaderExec (#31230) (#31235)
e2ab8a5 
close #31229
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@guo-shaoge guo-shaoge

Labels
affects-5.4 This bug affects 5.4.x versions. severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

executor: fix data race in IndexMergeReaderExec guo-shaoge/tidb
2 participants
@guo-shaoge @djshow832