-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Shard MySQL for connect four context #164
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Apply schema based sharding The shard is determined by a modulo operation. * Make database name configurable via env variable Doctrine commands aren't accepting a database name as a cli argument. By overwriting the env variable, see entrypoint, we can change the database without changing the whole connection string. * Acceptance tests are disabled for now This reason is that the number of shards isn't yet configurable.
marein
commented
Mar 9, 2023
marein
commented
Mar 9, 2023
marein
commented
Mar 9, 2023
* Besides the repository, the event store access must also be sharding aware, since this is read when the query model is not yet populated. For that matter, to query the events from the write model, the repository is used, which encapsulates the event store. Additionally, the event store is used directly for appending events. * Introduce ShardChooser ShardChooser encapsulates the logic of selecting a shard. * Remove query model's GameNotFoundException This is an aside refactoring, not related. This indirection isn't needed.
marein
force-pushed
the
119-shard-mysql-for-connect-four-context
branch
from
March 11, 2023 15:13
e9eda23
to
39776f3
Compare
marein
commented
Mar 12, 2023
marein
force-pushed
the
119-shard-mysql-for-connect-four-context
branch
from
March 14, 2023 22:28
c835b93
to
e46806c
Compare
Merged
They don't relate to this changes and might come in another pull request.
marein
force-pushed
the
119-shard-mysql-for-connect-four-context
branch
from
March 29, 2023 16:04
36fbfa2
to
cfd9adc
Compare
That didn't belong there and was just a quick implementation to serve the query model handler.
marein
force-pushed
the
119-shard-mysql-for-connect-four-context
branch
from
March 29, 2023 18:48
7644a78
to
c4860fd
Compare
marein
force-pushed
the
119-shard-mysql-for-connect-four-context
branch
from
March 31, 2023 15:49
5ead3a5
to
e5130e8
Compare
marein
added a commit
that referenced
this pull request
Apr 3, 2023
Applies schema-based sharding for the connect four database #119. The application itself only knows about logical shards and relies on a proxy, such as ProxySQL, to forward queries to physical shards. * The env variable APP_CONNECT_FOUR_DOCTRINE_DBAL_SHARDS defines a comma-separated list of active shards. As games can be sharded across multiple physical databases, the transactional scope (technical-wise) is moved to the repository layer, and removed from the application layer. The changes to the repository, which is now very persistence oriented, aim to make this explicit. The shard selection happens based on the game id. * The env variable APP_CONNECT_FOUR_DOCTRINE_DBAL_DATABASE is introduced to allow selecting a shard for database creation and migration, as doctrine commands currently don't allow passing the database name via cli arguments. * Besides the repository, the event store access must also be aware of sharding, since this is read when the query model is not yet populated. For that matter, to query the events from the write model, the repository is used, which encapsulates the EventStore. * Remove query model's GameNotFoundException This is an aside refactoring, not related. This indirection wasn't needed in the past. Additionally, an instance of GameId is required to find a game, not a loose string. * The current implementation doesn't take resharding into consideration.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull requests applies schema-based sharding to the connect four context. It plays nicely with ProxySQL (which will be added for the prod example later). The env variable
APP_CONNECT_FOUR_DOCTRINE_DBAL_URL
needs to point to ProxySQL instead of MySQL,APP_CONNECT_FOUR_DOCTRINE_DBAL_SHARDS
should define the logical shards andAPP_CONNECT_FOUR_DOCTRINE_DBAL_DATABASE
needs to point to the first shard. The following ProxySQL service definition routes 3 logical shards to 2 physical shards once #167 is merged.Show code
Tested with 3 physical MySQL shards and ProxySQL deployed as a sidecar of the app. The test was running for 10 minutes with a constant throughput of 25k write requests / s. The load was well distributed.
This pull request doesn't take resharding into consideration.