Benchmarks : Prune nightly benchmarks

[varun-sundar-rabindranath]

Summary:
The 2024-03-25 nightly benchmarks failed due to performance regressions.
We find that this is either due to,

• the inherent flakiness in the benchmark experiment itself (experiments with small work loads), or
• the inherent flakiness in the metrics.
  Please look at https://docs.google.com/document/d/1478BMToQIcpSCloiEWqmHoZVrVOZVV-1u4gCyqtjkKE/edit?usp=sharing for more details.

Updates in this PR:

• Serving case : Remove the 3000 num prompts at 10 qps experiments.
• Serving case : Mark the p90, p99 statistics as "Observation" metrics so they dont trigger failure.
• Engine case (benchmark_throughput.py) : Remove the 16 and 32 prefill cases.

Test:
Some local testing