[Observability Onboarding] Elasticsearch URL is pointing to Kibana on serverless - all onboarding flows

[flash1293]

Kibana version: serverless prod

Describe the bug: All flows on serverless use the Kibana URL instead of the Elasticsearch URL - this causes the flow to be unsuccessful.

Steps to reproduce:

1. Create Observability project on prod
2. Go to Auto-detect/Custom logs/Kubernetes flow
3. Elasticsearch endpoint is using the Kibana URL (<project-name>.kb.us-east-1.aws.elastic.cloud) instead of the Elasticsearch one <random id>.es.us-east-1.aws.elastic.cloud)

Expected behavior:
Should use the correct URL instead

Any additional context:

I didn't test but I think the problem comes from here:

kibana/x-pack/plugins/observability_solution/observability_onboarding/server/routes/logs/route.ts

Line 61 in 141e619

const elasticsearchUrl = plugins.cloud?.setup?.elasticsearchUrl

If the endpoint can't be fetched from the cloud endpoint, it will use the fallback (which is just the current Kibana host). I'm not sure what changed, maybe it's something within the cloud plugin or even the configuration for serverless projects. However, the integration page onboarding flow is not affected, so there is a proper way to get this information.

[elasticmachine]

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

[elasticmachine]

Pinging @elastic/obs-ux-onboarding-team (Feature: Observability Onboarding)

[akhileshpok]

@gbamparop - Can we please fix this issue before prioritising any feature development? Thanks

[gbamparop]

@gbamparop - Can we please fix this issue before prioritising any feature development? Thanks

Sure, let's tackle the high priority bugs first.