Skip to content

Conversation

@kylehuntsman
Copy link
Contributor

@kylehuntsman kylehuntsman commented Nov 3, 2022

Adds a booster-tester command, along with it's own image, for load testing piece store implementations. The goal is to be able to execute numerous graphsync and bitswap retrievals in parallel using known CIDs. Traces can then be viewed in the monitoring stack to view piece-store execution time length.

generate_cars_cmd is a WIP. Don't get payload CID output from the command, so I'm having to manually run boostx generate-car to have the details to make the deal command.

init.sh was to help initialize the container, but not as helpful as I originally thought since it can only really run two commands to setup the container to make deals.

@dirkmc
Copy link
Contributor

dirkmc commented Nov 4, 2022

This is a good start 👍

I think ideally we'd like to be able to run a script that we can use to load test bitswap and graphsync themselves, without needing to set up a devnet with miners etc.

I'd suggest starting with bitswap as booster-bitswap is already separated out into its own binary, so it should be simpler.
The command will likely look similar to runCmd in cmd/booster-bitswap/run.go

The command should

  • create a CAR file (or read one from the command line parameters)
  • create a PieceReader with a GetReader method that returns a reader over the CAR file
  • create a piecemeta service
  • create a piecemeta Client (just like in runCmd)
  • call NewPieceMeta to create a PieceMeta from the PieceReader and the piecemeta Client
  • call NewRemoteBlockstore with the PieceMeta to create a Blockstore
  • create a libp2p host
  • call NewBitswapServer with the libp2p host, and the Blockstore
  • start the bitswap server
  • create a bitswap client (see the booster-bitswap fetch command for an example)
  • execute a number of gets from the bitswap client to the bitswap server

For an example of how to create a piecemeta service, and how to create a CAR file programatically check out extern/boostd-data/svc/svc_test.go in #573

@dirkmc
Copy link
Contributor

dirkmc commented Jan 17, 2023

Closing as load testing is happening in a different repo now

@dirkmc dirkmc closed this Jan 17, 2023
@nonsense nonsense deleted the kylehuntsman/booster-tester branch January 17, 2023 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants