27
27
# Default: empty
28
28
region : us-west-1
29
29
30
- # AWS session token.
31
- # Default: empty (optional; local testing only)
32
- session_token : test
33
-
34
30
# AWS SQS endpoint.
35
31
# This parameter *is only required* if your SQS endpoint is *not* hosted on the AWS infrastructure.
36
32
# Always leave this empty if your queue is on AWS SQS.
@@ -90,7 +86,6 @@ You may also use this option if your service *does* support dynamic IAM, but you
90
86
sqs:
91
87
key: ASIAIOSFODNN7EXAMPLE
92
88
secret: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
93
- session_token: AQoDYXdzEJr... # optional; usually not required
94
89
# If in a different region *or* if outside AWS, provide the correct region.
95
90
region: eu-west-1
96
91
` ` `
@@ -166,13 +161,29 @@ jobs:
166
161
167
162
{% endcode %}
168
163
164
+ # # FIFO Queues
165
+
166
+ AWS supports [FIFO](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-fifo-queues.html)
167
+ queues, which guarantee order of delivery. In order for RoadRunner to correctly handle this scenario, some
168
+ configuration is required.
169
+
170
+ 1. You should set `retain_failed_jobs` to `true` on the pipeline, so a failed job does not get re-queued at the back of
171
+ the queue. This would break the FIFO order.
172
+ 2. You must have either `num_workers` set to `1` for the worker pool **or** `prefetch` set to `1` for the pipeline. If
173
+ you have multiple workers and fetch multiple messages at a time (`prefetch > 1`), RoadRunner may not process the jobs in
174
+ correct order.
175
+
169
176
# # Configuration Options
170
177
171
178
# ## Prefetch
172
179
173
- ` prefetch` - Number of jobs to prefetch from the queue until ACK/NACK.
180
+ ` prefetch` - The number of jobs to request from SQS at a time. This sets the `MaxNumberOfMessages` parameter on the
181
+ [`ReceiveMessage`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html) call
182
+ to the queue. Additionally, it limits the number of queues that may be retained on the priority queue in RoadRunner for
183
+ the pipeline. If you set this to zero or omit it, the default value of 1 will be used.
174
184
175
- Default : ` 10`
185
+ Default : ` 1`
186
+ Maximum : ` 10`
176
187
177
188
# ## Visibility Timeout
178
189
@@ -182,8 +193,8 @@ job may run multiple times. If you do not provide a value for this parameter or
182
193
attribute of the queue is used, which defaults to 30 seconds. For more information, see the documentation on
183
194
[visibility timeouts](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html).
184
195
185
- Max value is `43200` seconds (12 hours).
186
196
Default : ` 0`
197
+ Maximum : ` 43200` (12 hours)
187
198
188
199
# ## Error Visibility Timeout
189
200
@@ -192,17 +203,18 @@ failed) jobs will have their visibility timeout changed to. Note that you cannot
192
203
more than the maximum value in cases where it fails multiple times, so setting this value to the maximum is error-prone.
193
204
This parameter is ignored if `retain_failed_jobs` is `false`, and it must be larger than zero to apply.
194
205
195
- Max value is `43200` seconds (12 hours).
196
206
Default : ` 0`
207
+ Maximum : ` 43200` (12 hours)
197
208
198
209
# ## Retain Failed Jobs
199
210
200
- ` retain_failed_jobs` - If enabled, jobs will not be deleted and requeued if they fail. Instead, RoadRunner will
211
+ ` retain_failed_jobs` - If enabled, jobs will ** not** be deleted and re-queued if they fail. Instead, RoadRunner will
201
212
simply let them be processed again after `visibility_timeout` has passed. If you set `error_visibility_timeout` and
202
213
enable this feature, RoadRunner will change the timeout of the job to the value of `error_visibility_timeout`. This lets
203
214
you customize the timeout for errors specifically. If you enable this feature, you can configure SQS to automatically
204
215
move jobs that fail multiple times to a
205
216
[dead-letter queue](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html).
217
+ Always enable this feature if you consume from a FIFO queue.
206
218
207
219
Default : ` false`
208
220
@@ -211,20 +223,22 @@ Default: `false`
211
223
` wait_time_seconds` - The duration (in seconds) for which the call waits for a message to arrive in the queue before
212
224
returning. If a message is available, the call returns sooner than `wait_time_seconds`. If no messages are available and
213
225
the wait time expires, the call returns successfully with an empty list of messages. Please note that this parameter
214
- cannot be explicitly configured to use zero, as zero will apply the queue defaults.
226
+ cannot be explicitly configured to use zero, as zero will apply the queue defaults. In most cases, you should set this
227
+ to a non-zero value.
215
228
216
229
Default : ` 0`
230
+ Maximum : ` 20`
217
231
218
232
{% hint style="warning" %}
219
- # ## Shor vs. Long Polling
233
+ # ## Short vs. Long Polling
220
234
221
235
By default, SQS and RoadRunner is configured to use **short polling**. Please review the documentation on the
222
236
differences between
223
237
[short and long polling](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-short-and-long-polling.html)
224
238
and make sure that your queue and RoadRunner is configured correctly. Long polling will *usually* be a cost-saving
225
239
feature with no practical impact on performance or functionality. Remember that not providing a value (or zero) for
226
- ` wait_time_seconds` will cause your queue polling to be based on the `ReceiveMessageWaitTimeSeconds` attribute
227
- configured on the queue.
240
+ ` wait_time_seconds` will cause your queue wait time to be based on the `ReceiveMessageWaitTimeSeconds` attribute
241
+ configured on the queue, which also defaults to zero .
228
242
{% endhint %}
229
243
230
244
# ## Queue
@@ -247,7 +261,7 @@ More info:
247
261
[link](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html#SQS-SendMessage-request-MessageGroupId)
248
262
249
263
# ## Skip Queue Declaration
250
- " https://sqs.eu-central-1.amazonaws.com "
264
+
251
265
` skip_queue_declaration` - By default, RR tries to create the queue (using the `queue` name) if it does not exist. Set
252
266
this option to `true` if the queue already exists.
253
267
@@ -284,8 +298,8 @@ Attributes are only set if RoadRunner creates the queue. Attributes of existing
284
298
285
299
# ## Tags
286
300
287
- ` tags` - Tags don't have any semantic meaning. Amazon SQS interprets tags as character. Tags are only set if RR creates
288
- the queue. Existing queues are not modified.
301
+ ` tags` - Tags don't have any semantic meaning. Amazon SQS interprets tags as character. Tags are only set if RoadRunner
302
+ creates the queue. Existing queues are ** not** modified.
289
303
290
304
{% hint style="info" %}
291
305
This functionality is rarely used and slows down queues. Please see
0 commit comments