rfc: AI gateway API

[daixiang0]

Ref opea-project/GenAIInfra#197

[tedivm]

I really think we should look at the open source Kong AI Gateway for this, as it has a lot of fantastic functionality.

[daixiang0]

I really think we should look at the open source Kong AI Gateway for this, as it has a lot of fantastic functionality.

As an open-source ecosystem orchestration framework, OPEA should not bind one gateway.

The goal of OPEA is to accelerate AI land rather than bringing new cost like learn/maintain new gateway, a lot of enterprise has their own gateway solution team, migration is a cost. It can help enterprise minimize investment and maximize use of existing resources.

Kong open source gateway is good, but some powerful features are not open source, if we bind Kong, we need to find way to implement them while a new gateway implement is not our goal.

So the AI Gateway API provides a abstraction layer to use popular gateway solution, like Kong, Envoy, APISIX or others.

At the same time, it would get more potential partners who has their own AI gateway solution.

[ftian1]

I will merge this PR. if we have further discussion, we can refine this one.

[daixiang0]

@ftian1 thanks