Add page abount flow control #83
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mat-hek
requested changes
Sep 25, 2023
|
|
||
| - output pad with `flow_control: :manual` - the element gets the information, through the `handle_demand` callback, how much data a specific output pad needs. An element having an output pad with `:manual` `flow control` is obligated to satisfy the demand on it. | ||
|
|
||
| - input pad with `flow_control: :auto` - since `v1.0.0-rc1` the default `flow control` value. Unlike in manual demands, the element must not directly specify the demand value on the input pad to receive buffers on it. In `:auto` `flow control`, the framework will manage the demand value itself. In the case of Sources and Endpoints, it will make sure that the demand on the input pad is positive as long as the number of buffers waiting for processing in the mailbox is not too large. In the case of filters, Membrane will take care of a positive demand on under one additional condition - the demand value on all output pads with `flow_control: :auto` of this element must also be positive. |
There was a problem hiding this comment.
I wouldn't mention the new defaults until we have 1.0
There was a problem hiding this comment.
Suggested change
| - input pad with `flow_control: :auto` - since `v1.0.0-rc1` the default `flow control` value. Unlike in manual demands, the element must not directly specify the demand value on the input pad to receive buffers on it. In `:auto` `flow control`, the framework will manage the demand value itself. In the case of Sources and Endpoints, it will make sure that the demand on the input pad is positive as long as the number of buffers waiting for processing in the mailbox is not too large. In the case of filters, Membrane will take care of a positive demand on under one additional condition - the demand value on all output pads with `flow_control: :auto` of this element must also be positive. | |
| - input pad with `flow_control: :auto` - since `v1.0.0-rc1` the default `flow control` value. Unlike in manual demands, the element doesn't have to directly specify the demand value on the input pad to receive buffers on it. In `:auto` `flow control`, the framework will manage the demand value itself. In the case of Sources and Endpoints, it will make sure that the demand on the input pad is positive as long as the number of buffers waiting for processing in the mailbox is not too large. In the case of filters, Membrane will take care of a positive demand on under one additional condition - the demand value on all output pads with `flow_control: :auto` of this element must also be positive. |
|
|
||
| - output pad with `flow_control: :push` - the element decides for itself when to send how many buffers via a given pad. | ||
|
|
||
| - input pad with `flow_control: :manual` - the element directly, using the `demand` action, determines how much data it wants to receive from this pad. Membrane guarantees that more data will never come to the input pad than was demanded.. |
There was a problem hiding this comment.
Suggested change
| - input pad with `flow_control: :manual` - the element directly, using the `demand` action, determines how much data it wants to receive from this pad. Membrane guarantees that more data will never come to the input pad than was demanded.. | |
| - input pad with `flow_control: :manual` - the element directly, using the `demand` action, requests how much data it wants to receive from this pad. Membrane guarantees that more data will never come to the input pad than was demanded. |
|
|
||
| - input pad with `flow_control: :auto` - since `v1.0.0-rc1` the default `flow control` value. Unlike in manual demands, the element must not directly specify the demand value on the input pad to receive buffers on it. In `:auto` `flow control`, the framework will manage the demand value itself. In the case of Sources and Endpoints, it will make sure that the demand on the input pad is positive as long as the number of buffers waiting for processing in the mailbox is not too large. In the case of filters, Membrane will take care of a positive demand on under one additional condition - the demand value on all output pads with `flow_control: :auto` of this element must also be positive. | ||
|
|
||
| - output pad with `flow_control: :output` - an auto output pad does not require from the element to implement the `handle_demand` callback. The demand value on this pad only matters when the framework itself calculates the value of the auto demand on the input pads. |
There was a problem hiding this comment.
Suggested change
| - output pad with `flow_control: :output` - an auto output pad does not require from the element to implement the `handle_demand` callback. The demand value on this pad only matters when the framework itself calculates the value of the auto demand on the input pads. | |
| - output pad with `flow_control: :auto` - an auto output pad does not require from the element to implement the `handle_demand` callback. The demand value on this pad only matters when the framework itself calculates the value of the auto demand on the input pads. |
|
|
||
| - input pad with `flow_control: :manual` - the element directly, using the `demand` action, determines how much data it wants to receive from this pad. Membrane guarantees that more data will never come to the input pad than was demanded.. | ||
|
|
||
| - output pad with `flow_control: :manual` - the element gets the information, through the `handle_demand` callback, how much data a specific output pad needs. An element having an output pad with `:manual` `flow control` is obligated to satisfy the demand on it. |
There was a problem hiding this comment.
Suggested change
| - output pad with `flow_control: :manual` - the element gets the information, through the `handle_demand` callback, how much data a specific output pad needs. An element having an output pad with `:manual` `flow control` is obligated to satisfy the demand on it. | |
| - output pad with `flow_control: :manual` - the element gets the information, through the `handle_demand` callback, how much data a specific output pad needs. An element having an output pad with `flow control: :manual` is obligated to satisfy the demand on it. |
There was a problem hiding this comment.
Also, let's add an example of when manual demands are needed
|
|
||
| When defining a pad od an element, you can pass one of 3 values in the `flow_control` field: `auto`, `push`, or `manual`. They determine how the backpressure mechanism works on the pad they relate to. | ||
|
|
||
| - input pad with `flow_control: :push` - an element with such a pad must always be ready to receive a new buffer on that pad. The output pad linked with such a input pad must also have `flow control` set to `:push`. |
There was a problem hiding this comment.
Let's start with capital letters
|
|
||
| When defining a pad od an element, you can pass one of 3 values in the `flow_control` field: `auto`, `push`, or `manual`. They determine how the backpressure mechanism works on the pad they relate to. | ||
|
|
||
| - input pad with `flow_control: :push` - an element with such a pad must always be ready to receive a new buffer on that pad. The output pad linked with such a input pad must also have `flow control` set to `:push`. |
There was a problem hiding this comment.
I'd mention that it's usually not advised to use push input, as it's prone to overflow
FelonEkonom
force-pushed
the
get-started-with-flow-control
branch
2 times, most recently
from
baaa645 to
46f829d
Compare
mat-hek
requested changes
Oct 3, 2023
| - `manual`, these buffers will wait in line until the element raises the demand on its input by returning the `demand` action. | ||
| - `auto`, these buffers will be processed, and the demand value on the input pad will drop below zero. | ||
|
|
||
| In both of the above cases, when much more data is sent than was demanded on the input pad, a `toilet overflow` error will be thrown. It means that the element with the push output pad sends more data than the rest of the pipeline can process. We can control the limit value, above which the `toilet overflow` error will be thrown, using the `toilet_capacity` option passed to the `via_in/3` function when creating the spec, e.g. |
There was a problem hiding this comment.
This is only true if the effective flow control is pull
mat-hek
approved these changes
Oct 16, 2023
FelonEkonom
force-pushed
the
get-started-with-flow-control
branch
from
6337ae7 to
35d9bbe
Compare
FelonEkonom
force-pushed
the
get-started-with-flow-control
branch
from
35d9bbe to
d5f5dfc
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Jira ticket: https://membraneframework.atlassian.net/browse/MC-231