0003: Smart Routing

[mkrueger12]

Add a pool that is the "smart" routing pool. Effectively a pool that classifies prompts and sends them to the optimal endpoint. Prompts could be classified via any number of categories.

For example, say that a prompt comes in that is best answered by a specialized, legal endpoint. The smart pool classifies and routes it to the legal endpoint.

Likely involves traditional ML.

[roma-glushko]

This is a great feature to have! I would consider it post-MVP but we surely could think of it right now.

[mkrueger12]

I agree on the timing as well. For sure post-MVP.