Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MONGOSH-400 - Use mlaunch for spawning test mongod processes #401

Merged
merged 2 commits into from
Nov 17, 2020

Conversation

addaleax
Copy link
Contributor

This is meant to finish up MONGOSH-400, i.e. investigating whether we can use mlaunch, rather than providing an actual working implementation, although this already works quite well imo. Evergreen Windows hosts apparently (?) do not have Python 3.x installed, that’s something we might need to solve before this could theoretically be merged. All in all, I’m pretty happy with how mlaunch itself works and the capabilities we get from it, in particular, it does enable running the replica set/sharding tests 🙂


Switch to using mlaunch for spwaning mongod processes
and enable the replica set and sharding tests.

If merged in its current form, this would resolve

@addaleax addaleax added the wip Work in Progress label Nov 11, 2020
@addaleax addaleax force-pushed the mlaunch branch 2 times, most recently from 5926734 to d6a4b7a Compare November 11, 2020 18:18
['--single', '--replSet', replId],
['--single', '--replSet', replId]
);
let cfg: {_id: string, members: {_id: number, host: string, priority: number}[]};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so cool I haven't seen this syntax before!

Copy link
Contributor

@aherlihy aherlihy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Nice job

internalState = new ShellInternalState(serviceProvider);
mongo = internalState.currentDb.getMongo();
rs = new ReplicaSet(mongo);

// check replset uninitialized
try {
await rs.status();
expect.fail();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So mlaunch supports starting up uninitialized replsets? If it's not possible then we can potentially just skip the init test for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aherlihy Yes, the tests here run & pass :) It doesn’t have a designated feature for starting uninitialized replsets, but you can pass pretty much any mongod server option through over it

@addaleax addaleax force-pushed the mlaunch branch 11 times, most recently from 6b56dea to 4e76028 Compare November 12, 2020 16:41
Copy link
Contributor

@rose-m rose-m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks amazing and seems to make those tests super easy! I tried running it locally, however, and found we probably have to add 2 other PIP packages (see my comment) to make it work out of the box 👍

testing/integration-testing-hooks.ts Show resolved Hide resolved
@rose-m
Copy link
Contributor

rose-m commented Nov 13, 2020

Ah and there's one failing test:

@mongosh/shell-api:   1) Shard
@mongosh/shell-api:        integration
@mongosh/shell-api:          sharding info
@mongosh/shell-api:            returns the status:
@mongosh/shell-api:       AssertionError: expected [ Array(6) ] to be a superset of [ Array(6) ]
@mongosh/shell-api:       + expected - actual
@mongosh/shell-api:        [
@mongosh/shell-api:          "shardingVersion"
@mongosh/shell-api:          "shards"
@mongosh/shell-api:       -  "most recently active mongoses"
@mongosh/shell-api:       +  "active mongoses"
@mongosh/shell-api:          "autosplit"
@mongosh/shell-api:          "balancer"
@mongosh/shell-api:          "databases"
@mongosh/shell-api:        ]
@mongosh/shell-api:
@mongosh/shell-api:       at Context.<anonymous> (src/shard.spec.ts:1140:54)

@addaleax addaleax force-pushed the mlaunch branch 2 times, most recently from cededad to b08f3c8 Compare November 13, 2020 14:39
@addaleax
Copy link
Contributor Author

Ready for review, assuming that I didn’t manage to regress into breaking CI :)

Copy link
Contributor

@rose-m rose-m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried it again - and worked like a charm on my machine 👍 Code (as far as I know the context and understood it) looks good to me 😄

Switch to using `mlaunch` for spwaning mongod processes
and enable the replica set and sharding tests.

If merged in its current form, this would resolve

- MONGOSH-400 (Decide how to spawn mongod processes)
- MONGOSH-411 (Switch mongosh tests to use mlaunch)
- MONGOSH-420 (Enable sharding tests)
- MONGOSH-421 (Enable replica set tests)
Copy link
Contributor

@aherlihy aherlihy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me! Nice job, this is a huge improvement :) I wonder if it' would be possible to silence the mlaunch 'Running command' printouts for every test? Unless you think it's useful to see. Feel free to merge right away.

return new URL(this.connectionString()).hostname;
}

port(): string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should update this to be something other than 27017?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand -- the default port for mongodb:// URLs is 27017, so we should return that if there's no port in the URL, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - that's right. I was thinking maybe we'd want to prevent tests from starting a server on 27017 (since most people have something else running on 27017 so if you start your test processes it may cause collisions), but it looks like none of the tests are doing that anyway and it would be incorrect/unexpected to always avoid 27017. So ignore my comment!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general idea here is that you can still specify MONGOSH_TEST_SERVER_URL for debugging if that helps to point to a specific server, and that in that case, if that URL does not have a port, we should use 27017 :)

@addaleax
Copy link
Contributor Author

I wonder if it' would be possible to silence the mlaunch 'Running command' printouts for every test? Unless you think it's useful to see. Feel free to merge right away.

Hm yeah ... it's been helpful for debugging this PR itself, but I can remove them/comment them out until we run into problems with these 👍

@addaleax addaleax merged commit 4d4dd94 into master Nov 17, 2020
@addaleax addaleax deleted the mlaunch branch November 17, 2020 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wip Work in Progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants