-
-
Notifications
You must be signed in to change notification settings - Fork 5.7k
Implement and export isfull
#53159
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement and export isfull
#53159
Conversation
|
@KronosTheLate Thanks for the shoutout! |
base/channels.jl
Outdated
| Note that if another task is currently waiting to `take!` | ||
| a value from `c`, a call to `put!(c, some_value)` might | ||
| immediately be unblocked, even if `isfull(c)==true`. | ||
| This is particularly relevant for unbuffered channels. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some possible word-smithing, to try to teach a user why they should never use this method(except of course, for cases where this query is useful):
| Note that if another task is currently waiting to `take!` | |
| a value from `c`, a call to `put!(c, some_value)` might | |
| immediately be unblocked, even if `isfull(c)==true`. | |
| This is particularly relevant for unbuffered channels. | |
| Note that it may be frequently the case that `put!` will not block after this returns | |
| `true`. Users must take precautions not to accidentally create live-lock bugs in their code | |
| by calling this method, as these are generally harder to debug than deadlocks. It is also | |
| possible that `put!` will block after this call, if there are multiple producer tasks | |
| calling `put!` in parallel. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While this looks like a good warning, it seems odd to me to describe the function as "test if full in the sense that put will block", and in the next sentence say that "maybie frequently this will ont block when returning true". If I read that, I would be left with a feeling of "okay, so which one is it". In that sense I like my original version, that just explains a case in which it might go wrong, and leaves it to the user to use that knowledge.
But the warning could be harder. What about
Note that a call to
isfullgenerally occurs in a different instant than any call toput!. In particular, callingisfulland then subsequentlyput!guarantees that the calls occur at different instants. Therefore, the state of the channel may change between the calls, and generallyisfullcan not safely be used to see if a call toput!will in fact block.
A little verbose, but very clear and understandable IMO.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Your original version uses some words that have specific meanings for thread atomics, and those words don't apply in their defined technical sense (in particular "currently" is a term that seems to imply seq-cst guarantees, which does not hold here)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The word "instant" is also a seq-cst term, and does not apply here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ment to say a different point in time. Would that be better?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just read my latest suggestion again, and I like it a lot. It is short, precise, and understandable.
EDIT: But this would hardly be the first time I am overly happy with a phrasing of mine that may be illegible to others, so I am very curious about your opinion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are describing seq-cst, which is an ordering that exists, and is the strictest ordering, but is not the ordering that applies here. Events do have temporal coordinates, and they do have a time at that temporal coordinate, but that does not imply anything about the sequence in which events happen at other coordinates in space.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So the problem with that warning is that the state of the channel also might have already been changed before isfull, but it wasn't observed until afterwards on this thread, as the memory ordering here is simply :monotonic. Or consider instead if there is another producer thread, it might already have the lock to insert a new value, but has not updated n_avail yet. Thus isfull is false, but the channel is already full. The :monotonic ordering forbids unstable observations (if a thread observes a value, then it will continue to observe that value until it is changed), but in isolation it doesn't enforce any other useful properties--that requires holding the channel lock. The put! and take! have acquire-release ordering, so that combination adds at least some additional properties in combination, but it is unclear if they are useful properties, unless you are writing stochastic algorithms (which is quite reasonable with threading to use algorithms that are resilient to such behaviors, such as exponential backoff).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am surely in over my head regarding the details you are describing. If you were able to write something in roughly the format of my latest suggestion, but correct, I think that might be best. Otherwise, your original proposal is alright by me, though harder to read than would be ideal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some minor tweaks to your original proposal:
Note that it may frequently be the case that `put!` will
not block after this returns `true`. Users must take
precautions not to accidentally create live-lock bugs
in their code by calling this method, as these are
generally harder to debug than deadlocks. It is also
possible that `put!` will block after this call
returns `false`, if there are multiple producer
tasks calling `put!` in parallel.
That is looks great by me, specially considering how far into the weeds the conversation goes when I try to be more precise xD I have committed this version of your original proposal.
vtjnash
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some doc comments, but otherwise LGTM
Co-authored-by: Jameson Nash <vtjnash@gmail.com>
Co-authored-by: Jameson Nash <vtjnash@gmail.com>
|
Bump |
This PR implements `isfull(c::Channel)`. It calls `n_avail(c) ≥ c.sz_max` in all cases. The original implementation was inspired by [this comment](https://discourse.julialang.org/t/function-to-check-if-channel-is-full/44795/3?u=thelatekronos), and therefore had a special case for unbuffered channels, which fell back to `isready`. I opted against this behaviour, because it fails to respect that an unbuffered channel is always full, in two important senses: 1) The number of elements available is greater than or equal the capacity 2) A call to `put!` will block With the current implementation, the behaviour is simply understood and summarized in all cases by the start of the docstring: > Determines whether a `Channel` is full, in the sense that calling `put!(c, some_value)` will block. Shoutout to @SamuraiAku for their work in JuliaLang#40720, which helped me a lot on thinking this through, and remembering to change all relevant files. In particular, the detail around how `c.cond_take.waitq` may result in immediate unblocking, which is a really important caveat on a function that may be used to check if `put!`ing will block. However, for buffered channels, `isfull` is extremely close to `putwillblock` from JuliaLang#40720 (just a little better, with >= instead of ==), and for unbuffered channels it does not make much sense to see if `put!`ing will block. This PR is created based on [this](JuliaLang#22863 (comment)) "call to action". Checklist: - [x] Entry in news - [x] Docstring with example - [x] Export function - [x] Mention in manual - [x] Entry in [docs-reference](https://docs.julialang.org/en/v1/base/parallel/) --------- Co-authored-by: Jameson Nash <vtjnash@gmail.com>
|
I would not find the changes made here in the 1.11 alpha release notes. Is further action required for this to make it into there? |
|
(FWIW, I think this should have been / should be more descriptive. |
Fair point. I expect that many people who regularly work with matrices, which are a lot of people, would feel the same. Importantly, people who use Julia are more likely to also be people who regularly work with matrices. So I expect this will be an issue for others as well. I do however have to object. The opposite of full is empty, not sparse. Full/empty is a more natural dichotomy than full/sparse. The current name has the most natural meaning, without being so specific as to become hard to discover. Also, would the opposite of sparse not be dense? I may be wrong of course, but I feel quite certain that the name is fine. |
|
Yeah, thinking about it some more, I guess |
This PR implements `isfull(c::Channel)`. It calls `n_avail(c) ≥ c.sz_max` in all cases. The original implementation was inspired by [this comment](https://discourse.julialang.org/t/function-to-check-if-channel-is-full/44795/3?u=thelatekronos), and therefore had a special case for unbuffered channels, which fell back to `isready`. I opted against this behaviour, because it fails to respect that an unbuffered channel is always full, in two important senses: 1) The number of elements available is greater than or equal the capacity 2) A call to `put!` will block With the current implementation, the behaviour is simply understood and summarized in all cases by the start of the docstring: > Determines whether a `Channel` is full, in the sense that calling `put!(c, some_value)` will block. Shoutout to @SamuraiAku for their work in JuliaLang#40720, which helped me a lot on thinking this through, and remembering to change all relevant files. In particular, the detail around how `c.cond_take.waitq` may result in immediate unblocking, which is a really important caveat on a function that may be used to check if `put!`ing will block. However, for buffered channels, `isfull` is extremely close to `putwillblock` from JuliaLang#40720 (just a little better, with >= instead of ==), and for unbuffered channels it does not make much sense to see if `put!`ing will block. This PR is created based on [this](JuliaLang#22863 (comment)) "call to action". Checklist: - [x] Entry in news - [x] Docstring with example - [x] Export function - [x] Mention in manual - [x] Entry in [docs-reference](https://docs.julialang.org/en/v1/base/parallel/) --------- Co-authored-by: Jameson Nash <vtjnash@gmail.com>
This PR implements
isfull(c::Channel).It calls
n_avail(c) ≥ c.sz_maxin all cases. The original implementation was inspired by this comment, and therefore had a special case for unbuffered channels, which fell back toisready. I opted against this behaviour, because it fails to respect that an unbuffered channel is always full, in two important senses:1) The number of elements available is greater than or equal the capacity
2) A call to
put!will blockWith the current implementation, the behaviour is simply understood and summarized in all cases by the start of the docstring:
Shoutout to @SamuraiAku for their work in #40720, which helped me a lot on thinking this through, and remembering to change all relevant files. In particular, the detail around how
c.cond_take.waitqmay result in immediate unblocking, which is a really important caveat on a function that may be used to check ifput!ing will block. However, for buffered channels,isfullis extremely close toputwillblockfrom #40720 (just a little better, with >= instead of ==), and for unbuffered channels it does not make much sense to see ifput!ing will block.This PR is created based on this "call to action".
Checklist: