Long polling in BroadwaySQS
By default BroadwaySQS doesn’t wait for a message to arrive if no messages are available in the queue. Also, it has a default 5 seconds duration for which the producer waits before making a request for more messages. This may result in few second’s gap between enqueuing a message and processing it. For many cases this is good enough, but sometimes one may need a message to be processed immediately after enqueueing. In such cases you should consider using long polling.
To achieve it in BroadwaySQS just set wait_time_seconds
to e.g. 10
seconds (or any value from 1 to 20) and receive_interval
to 0
in producer’s options.
Example:
def start_link(_opts) do
producer = Application.get_env(:scanner, :broadway_producer, BroadwaySQS.Producer)
queue_url = Application.get_env(:ex_aws_sqs, :queue_url)
Broadway.start_link(__MODULE__,
name: __MODULE__,
producers: [
default: [
module: {
producer,
queue_url: queue_url, wait_time_seconds: 20, receive_interval: 0
},
stages: 1
]
],
processors: [
default: [stages: 50]
]
)
end
Tweet