Stress test available LLM deployment serve using FastAPI.
It is possible to use Docker for this directory, but the development is too slow.
- RTX 3090 Ti, underwatt 450W, slotted at first slot, PCIe 5.0 x16, motherboard https://www.msi.com/Motherboard/MPG-Z690-EDGE-WIFI-DDR4/Specification
- i7-12700 Processor, Up to 1x16+4, 2x8+4 PCI Express Configurations.
- 64GB RAM, 2 slots, each 32GB RAM DDR4 2400 MT/s.