Update swirl #2269
Merged
Update swirl #2269
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This removes some of the boilerplate we had to add last night for manually managing connection pools with swirl. Swirl now can automatically manage its own connection pool. It can be given either a database URL, a pool builder, or a fully built pool (which is used in tests). If it's given one of the first two, it will configure the connection pool to have a size of 2x the thread count. I've also let it go up to the default number of threads, which is the number of CPU cores. Since this would mean the pool size grows pretty dramatically, I've configured it to not try to keep any idle connections (meaning it will try to establish the connection when asked for one, not before). Additionally, we no longer need the connection pool to live on our environment, we can take either a connection or a pool as an argument to a background job, and swirl will automatically take it from the pool. In the future I'm going to try and have swirl make some more intelligent decisions on this, so we could do something like have 2 threads only run jobs that don't need a connection (like publish!) Heroku doesn't guarantee the number of logical cores on standard dynos, but it appears to have consistently been 4 for a while. This means we'll be doubling the number of background workers, and potentially using up to 8 database connections on worker. Now that we've moved some web traffic to the replica, we may want to lower the pool size for the primary db on the web servers to compensate.
|
r? @smarnach (rust_highfive has picked a reviewer for you, use r? to override) |
Closed
|
LGTM @bors r+ |
|
📌 Commit 93c46d3 has been approved by |
bors
added a commit
that referenced
this pull request
Mar 22, 2020
Update swirl This removes some of the boilerplate we had to add last night for manually managing connection pools with swirl. Swirl now can automatically manage its own connection pool. It can be given either a database URL, a pool builder, or a fully built pool (which is used in tests). If it's given one of the first two, it will configure the connection pool to have a size of 2x the thread count. I've also let it go up to the default number of threads, which is the number of CPU cores. Since this would mean the pool size grows pretty dramatically, I've configured it to not try to keep any idle connections (meaning it will try to establish the connection when asked for one, not before). Additionally, we no longer need the connection pool to live on our environment, we can take either a connection or a pool as an argument to a background job, and swirl will automatically take it from the pool. In the future I'm going to try and have swirl make some more intelligent decisions on this, so we could do something like have 2 threads only run jobs that don't need a connection (like publish!) Heroku doesn't guarantee the number of logical cores on standard dynos, but it appears to have consistently been 4 for a while. This means we'll be doubling the number of background workers, and potentially using up to 8 database connections on worker. Now that we've moved some web traffic to the replica, we may want to lower the pool size for the primary db on the web servers to compensate.
|
@bors r+ |
|
📌 Commit 893c5e3 has been approved by |
|
☀️ Test successful - checks-travis |
Merged
sgrif
added a commit
to sgrif/crates.io
that referenced
this pull request
Apr 16, 2020
This is a second attempt at rust-lang#2269, which had to be reverted shortly after being deployed, as it caused jobs to fail to run in production. This commit only differs from that PR in the revision of swirl used. The cause of the issue was the change in the signature of `update_downloads`, which does not need anything from the job environment, and so it only takes a connection. This changed the environment type for that job from `background_jobs::Environment` to `()`, so our runner no longer knew how to run that job. The smallest fix in crates.io would have been to add an unused environment argument to `update_downloads`. However, having some jobs not require a shared environment felt common enough that I fixed this in swirl instead. sgrif/swirl#23 changed the behavior so jobs with no environment (a.k.a. the environment type is `()`) are always run, regardless of the environment type of the runner. Since `update_downloads` is essentially a cron job, and is not enqueued from the web server directly, this was not caught by our integration suite. This case was caught quickly after being deployed, and only affected a non-critical part of the service, so I've opted not to figure out how to add this to our integration suite.
Merged
bors
added a commit
that referenced
this pull request
May 2, 2020
Update Swirl, take two This is a second attempt at #2269, which had to be reverted shortly after being deployed, as it caused jobs to fail to run in production. This commit only differs from that PR in the revision of swirl used. The cause of the issue was the change in the signature of `update_downloads`, which does not need anything from the job environment, and so it only takes a connection. This changed the environment type for that job from `background_jobs::Environment` to `()`, so our runner no longer knew how to run that job. The smallest fix in crates.io would have been to add an unused environment argument to `update_downloads`. However, having some jobs not require a shared environment felt common enough that I fixed this in swirl instead. sgrif/swirl#23 changed the behavior so jobs with no environment (a.k.a. the environment type is `()`) are always run, regardless of the environment type of the runner. Since `update_downloads` is essentially a cron job, and is not enqueued from the web server directly, this was not caught by our integration suite. This case was caught quickly after being deployed, and only affected a non-critical part of the service, so I've opted not to figure out how to add this to our integration suite. r? @jtgeibel
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This removes some of the boilerplate we had to add last night for
manually managing connection pools with swirl. Swirl now can
automatically manage its own connection pool. It can be given either a
database URL, a pool builder, or a fully built pool (which is used in
tests). If it's given one of the first two, it will configure the
connection pool to have a size of 2x the thread count. I've also let it
go up to the default number of threads, which is the number of CPU
cores. Since this would mean the pool size grows pretty dramatically,
I've configured it to not try to keep any idle connections (meaning it
will try to establish the connection when asked for one, not before).
Additionally, we no longer need the connection pool to live on our
environment, we can take either a connection or a pool as an argument to
a background job, and swirl will automatically take it from the pool. In
the future I'm going to try and have swirl make some more intelligent
decisions on this, so we could do something like have 2 threads only run
jobs that don't need a connection (like publish!)
Heroku doesn't guarantee the number of logical cores on standard dynos,
but it appears to have consistently been 4 for a while. This means we'll
be doubling the number of background workers, and potentially using up
to 8 database connections on worker. Now that we've moved some web
traffic to the replica, we may want to lower the pool size for the
primary db on the web servers to compensate.