Run platform functional tests with/without base-path

[mshustov]

FTR runs functional tests with --no-base-path option.

kibana/test/common/config.js

Line 43 in 5bc233f

'--no-base-path',

That is a source of many regressions during migration:
#54687
#56409
#57301
#62016
#63417 (review)

We would like to have reliable function tests to prevent such problems in the future.

Probably @elastic/kibana-operations can provide more information about the requirement to run FTR without a base path. As I can see this setting was on since the kbn-test package was introduced in #18568

I'm not sure we could and should enable configurable base-path for all the tests. Probably, we can start with moving the platform tests to a separate test config that runs tests with and without the base-path option.

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[spalger]

Yeah, one problem with testing with the base path turned on is that we test against the distributable, which doesn't include the base path proxy, so we would need to setup a separate proxy for use in the functional tests (which would probably be ideal since the base path proxy has conveniences for development that production proxies do not).

We would also need the tests to work against cloud Kibana instances that don't have base paths configured, so they can't rely on the base path being there, but that also shouldn't be a problem.

[mshustov]

Can we run Nginx as a reverse proxy when #68173 merged?

[spalger]

We'd probably need to add support for running the containers on the host network or something, but should be a pretty straightforward change to the dockerServers service. Only thing is that we want to make sure we don't make docker required to run most tests, so it'd be ideal if it ran as an alternate way to connect to the standard Kibana instance that most of the tests talk to.

[joshdover]

Only thing is that we want to make sure we don't make docker required to run most tests

Is this a firm decision? If we're finding success with Docker for test orchestration, I don't think we should hesitate to require it. It's quite a mature technology at this point and well supported on all platforms.

[spalger]

Only thing is that we want to make sure we don't make docker required to run most tests

Is this a firm decision? If we're finding success with Docker for test orchestration, I don't think we should hesitate to require it. It's quite a mature technology at this point and well supported on all platforms.

The reasons I have for not wanting to require docker are not that it's new or unsupported. I still feel like there are a number of usability gotchas, like debugging specific processes, image creation performance with massive directories like our node_modules, network management, mounted volume performance, etc. There are also maintenance considerations like GC, determining what's running, and less OS diversity in our development environments that I think might have a pretty negative impact on developers.

I know a lot of teams rely on docker, I've investigated moving several parts of the "operations experience" to Docker, and every time I have run into a roadblock that I makes things overall worse.

I'm still open minded to applying docker strategically, but I just want to express caution about requiring it without actually testing things out and making sure that it doesn't just make things conceptually simpler, but actually makes things better.

[spalger]

Sorry, I'm backing up my stance that we "make sure we don't make docker required", but requiring it for something small like nginx might be reasonable as most of the issues I described could be eliminated

• we're not building the image
• we'd only need to sync simple config files into the container
• debugging nginx isn't really a part of developing Kibana
• the container is small enough and would change infrequently enough that GC likely wouldn't be a constant problem
• OS diversity is still maintained as all Kibana code is running on the host machine directly

So from that angle I might have been a bit hasty about "not requiring docker" for something like an nginx proxy. That said, a module which implements a proxy (maybe based on any popular proxy library from npm) is a much simply way to accomplish this in my mind.

[mshustov]

@spalger is #76935 prerequisite for the current issue to decouple basePathProxy from the platform and ClusterManager?

[spalger]

I suppose, yeah

[mshustov]

I suppose, yeah

ok, I think we might use another nodejs server since we run FTR not in dev mode anyway