Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data race in module on "StageBidderRequest" #3576

Open
linux019 opened this issue Mar 15, 2024 · 2 comments
Open

Data race in module on "StageBidderRequest" #3576

linux019 opened this issue Mar 15, 2024 · 2 comments
Assignees
Labels

Comments

@linux019
Copy link
Contributor

linux019 commented Mar 15, 2024

I need to access bidder aliases in the RequestWrapper to resolve adapter name (as modules on many stages receive bidder alias, not "bidder core name" = adapter name)
race.log - full log
race.tar.gz - test scripts & config

  1. Use pbs.yaml with enabled default module:
external_url: "http://localhost:8000"

metrics:
  prometheus:
    port: 9100
auction_timeouts_ms:
  default: 60000
  max: 600000

compression:
  response:
    enable_gzip: false
  request:
    enable_gzip: false

gdpr:
  enabled: true
  default_value: "0"

adapters:
  ix:
    endpoint: https://00000.lb.indexww.com/pbs
    disabled: false
  rubicon:
    disabled: false
    endpoint: "https://exapi-us-east.rubiconproject.com/a/api/exchange"

hooks:
  enabled: true
  modules:
    prebid:
      ortb2blocking:
        enabled: true



  host_execution_plan:
    endpoints:
      "/openrtb2/auction":
        stages:

          bidder_request:
            groups:
              timeout: 100 #ms
              hook_sequence:
                - module_code: "prebid.ortb2blocking"
                  hook_impl_code: "prebid.ortb2blocking"
  1. Add following code here to read bidder aliases to the demo module
	if reqExt, err := payload.Request.GetRequestExt(); err == nil {
		_ = reqExt.GetPrebid().Aliases
	}

or apply patch with git apply file.patch (in the project root directory)
4. run server with enabled race detector go run -race main.go
5. use "run.sh" from the archive to run 50 parallel requests (it needs to run several times the shell script with curl to get a data race)
6. result

==================
WARNING: DATA RACE
Write at 0x00c000342c08 by goroutine 703:
  github.com/prebid/prebid-server/v2/openrtb_ext.(*RequestWrapper).GetRequestExt()
      github.com/prebid/prebid-server/v2/openrtb_ext/request_wrapper.go:127 +0xa4
  github.com/prebid/prebid-server/v2/modules/prebid/ortb2blocking.Module.HandleBidderRequestHook()
      github.com/prebid/prebid-server/v2/modules/prebid/ortb2blocking/module.go:25 +0x85
  github.com/prebid/prebid-server/v2/modules/prebid/ortb2blocking.(*Module).HandleBidderRequestHook()
      <autogenerated>:1 +0x16d
  github.com/prebid/prebid-server/v2/hooks/hookexecution.(*hookExecutor).ExecuteBidderRequestStage.func1()
      github.com/prebid/prebid-server/v2/hooks/hookexecution/executor.go:199 +0x161
  github.com/prebid/prebid-server/v2/hooks/hookexecution.executeHook[go.shape.interface { HandleBidderRequestHook(context.Context, github.com/prebid/prebid-server/v2/hooks/hookstage.ModuleInvocationContext, github.com/prebid/prebid-server/v2/hooks/hookstage.BidderRequestPayload) (github.com/prebid/prebid-server/v2/hooks/hookstage.HookResult[github.com/prebid/prebid-server/v2/hooks/hookstage.BidderRequestPayload], error) },go.shape.struct { Request *github.com/prebid/prebid-server/v2/openrtb_ext.RequestWrapper; Bidder string }].func1()
      github.com/prebid/prebid-server/v2/hooks/hookexecution/execution.go:108 +0x1e6

Previous read at 0x00c000342c08 by goroutine 576:
  github.com/prebid/prebid-server/v2/openrtb_ext.(*RequestWrapper).rebuildRequestExt()
      github.com/prebid/prebid-server/v2/openrtb_ext/request_wrapper.go:285 +0x3c
  github.com/prebid/prebid-server/v2/openrtb_ext.(*RequestWrapper).RebuildRequest()
      github.com/prebid/prebid-server/v2/openrtb_ext/request_wrapper.go:203 +0x9a
  github.com/prebid/prebid-server/v2/exchange.(*bidderAdapter).requestBid()
      github.com/prebid/prebid-server/v2/exchange/bidder.go:150 +0x24b
  github.com/prebid/prebid-server/v2/exchange.(*validatedBidder).requestBid()
      github.com/prebid/prebid-server/v2/exchange/bidder_validate_bids.go:35 +0x216
  github.com/prebid/prebid-server/v2/exchange.(*exchange).getAllBids.func1()
      github.com/prebid/prebid-server/v2/exchange/exchange.go:720 +0x98b
  github.com/prebid/prebid-server/v2/exchange.(*exchange).getAllBids.(*exchange).recoverSafely.func2()
      github.com/prebid/prebid-server/v2/exchange/exchange.go:829 +0x1c4
  github.com/prebid/prebid-server/v2/exchange.(*exchange).getAllBids.gowrap1()
      github.com/prebid/prebid-server/v2/exchange/exchange.go:751 +0x104

Goroutine 703 (running) created at:
  github.com/prebid/prebid-server/v2/hooks/hookexecution.executeHook[go.shape.interface { HandleBidderRequestHook(context.Context, github.com/prebid/prebid-server/v2/hooks/hookstage.ModuleInvocationContext, github.com/prebid/prebid-server/v2/hooks/hookstage.BidderRequestPayload) (github.com/prebid/prebid-server/v2/hooks/hookstage.HookResult[github.com/prebid/prebid-server/v2/hooks/hookstage.BidderRequestPayload], error) },go.shape.struct { Request *github.com/prebid/prebid-server/v2/openrtb_ext.RequestWrapper; Bidder string }]()
      github.com/prebid/prebid-server/v2/hooks/hookexecution/execution.go:105 +0x390
  github.com/prebid/prebid-server/v2/hooks/hookexecution.executeGroup[go.shape.interface { HandleBidderRequestHook(context.Context, github.com/prebid/prebid-server/v2/hooks/hookstage.ModuleInvocationContext, github.com/prebid/prebid-server/v2/hooks/hookstage.BidderRequestPayload) (github.com/prebid/prebid-server/v2/hooks/hookstage.HookResult[github.com/prebid/prebid-server/v2/hooks/hookstage.BidderRequestPayload], error) },go.shape.struct { Request *github.com/prebid/prebid-server/v2/openrtb_ext.RequestWrapper; Bidder string }].func1()
      github.com/prebid/prebid-server/v2/hooks/hookexecution/execution.go:78 +0x22c
  github.com/prebid/prebid-server/v2/hooks/hookexecution.executeGroup[go.shape.interface { HandleBidderRequestHook(context.Context, github.com/prebid/prebid-server/v2/hooks/hookstage.ModuleInvocationContext, github.com/prebid/prebid-server/v2/hooks/hookstage.BidderRequestPayload) (github.com/prebid/prebid-server/v2/hooks/hookstage.HookResult[github.com/prebid/prebid-server/v2/hooks/hookstage.BidderRequestPayload], error) },go.shape.struct { Request *github.com/prebid/prebid-server/v2/openrtb_ext.RequestWrapper; Bidder string }].gowrap1()
      github.com/prebid/prebid-server/v2/hooks/hookexecution/execution.go:79 +0xf8

Goroutine 576 (running) created at:
  github.com/prebid/prebid-server/v2/exchange.(*exchange).getAllBids()
      github.com/prebid/prebid-server/v2/exchange/exchange.go:751 +0x18f
  github.com/prebid/prebid-server/v2/exchange.(*exchange).HoldAuction()
      github.com/prebid/prebid-server/v2/exchange/exchange.go:378 +0x2624
  github.com/prebid/prebid-server/v2/endpoints/openrtb2.(*endpointDeps).Auction()
      github.com/prebid/prebid-server/v2/endpoints/openrtb2/auction.go:264 +0x1adb
  github.com/prebid/prebid-server/v2/endpoints/openrtb2.(*endpointDeps).Auction-fm()
      <autogenerated>:1 +0x73
  github.com/julienschmidt/httprouter.(*Router).ServeHTTP()
      github.com/julienschmidt/httprouter@v1.3.0/router.go:387 +0xf0b
  github.com/prebid/prebid-server/v2/router.(*Router).ServeHTTP()
      <autogenerated>:1 +0x64
  github.com/prebid/prebid-server/v2/router.SupportCORS.(*Cors).Handler.func2()
      github.com/rs/cors@v1.8.2/cors.go:231 +0x2f1
  net/http.HandlerFunc.ServeHTTP()
      net/http/server.go:2166 +0x47
  github.com/prebid/prebid-server/v2/router.NoCache.ServeHTTP()
      github.com/prebid/prebid-server/v2/router/router.go:119 +0x421
  github.com/prebid/prebid-server/v2/router.(*NoCache).ServeHTTP()
      <autogenerated>:1 +0x69
  net/http.serverHandler.ServeHTTP()
      net/http/server.go:3137 +0x2a1
  net/http.(*conn).serve()
      net/http/server.go:2039 +0x13c4
  net/http.(*Server).Serve.gowrap3()
      net/http/server.go:3285 +0x4f
==================
==================
WARNING: DATA RACE
Read at 0x00c00219dde0 by goroutine 841:
  github.com/prebid/prebid-server/v2/openrtb_ext.(*RequestWrapper).GetRequestExt()
      github.com/prebid/prebid-server/v2/openrtb_ext/request_wrapper.go:128 +0x10a
  github.com/prebid/prebid-server/v2/modules/prebid/ortb2blocking.Module.HandleBidderRequestHook()
@linux019
Copy link
Contributor Author

Moreover just running tests with enabled race detector go test -race ./... produces a lot of warnings in "exchange"

WARNING: DATA RACE
Read at 0x00c00053ae48 by goroutine 647:
  net/http.Header.Clone()
      /usr/lib/go/src/net/http/header.go:101 +0x5697
  github.com/prebid/prebid-server/v2/exchange.(*bidderAdapter).requestBid()
      exchange/bidder.go:174 +0x6b8
  github.com/prebid/prebid-server/v2/exchange.(*exchange).getAllBids.func1()
      exchange/exchange.go:733 +0x98b
  github.com/prebid/prebid-server/v2/exchange.(*exchange).getAllBids.(*exchange).recoverSafely.func2()
      exchange/exchange.go:842 +0x1c4
  github.com/prebid/prebid-server/v2/exchange.(*exchange).getAllBids.gowrap1()
      exchange/exchange.go:764 +0x104

Previous write at 0x00c00053ae48 by goroutine 648:
  net/http.Header.Clone()
      /usr/lib/go/src/net/http/header.go:114 +0x58d4
  github.com/prebid/prebid-server/v2/exchange.(*bidderAdapter).requestBid()
      exchange/bidder.go:174 +0x6b8

And all OpenRTB structures in https://github.com/prebid/prebid-server/blob/master/openrtb_ext/request_wrapper.go have accessors which modifies internal data on read access and this request wrapper is shared between adapters or hooks which are spawned in goroutines

@bretg bretg closed this as completed Mar 22, 2024
@SyntaxNode
Copy link
Contributor

Closed accidently during backlog grooming. Reopened.

@SyntaxNode SyntaxNode reopened this Mar 22, 2024
@bretg bretg added the PBS-Go label Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Needs Requirements
Development

No branches or pull requests

4 participants