Explanation of Sampling Param semantics in documentation

[mattf]

🚀 Describe the new functionality needed

Sampling params semantics

• strategy: greedy | top_k | top_p
• temperature: Optional[float] = 0.0
• top_p: Optional[float] = 0.95
• top_k: Optional[int] = 0

How does strategy relate to temperature / top_p / top_k?

What are the semantics when a user provides combinations of temperature / top_p / top_k?

💡 Why is this needed? What if we don't build it?

Users need to understand the impact of the SamplingParams on their inference requests.

Other thoughts

No response

[aidando73]

+1 - useful for us contributors as well so we know how the apis should be behaving. E.g., when adding a new provider

[hardikjshah]

We refactored sampling params so hopefully it should not we easy