Today I Learned

7 posts about #elixir

Long polling in BroadwaySQS

By default BroadwaySQS doesn’t wait for a message to arrive if no messages are available in the queue. Also, it has a default 5 seconds duration for which the producer waits before making a request for more messages. This may result in few second’s gap between enqueuing a message and processing it. For many cases this is good enough, but sometimes one may need a message to be processed immediately after enqueueing. In such cases you should consider using long polling.

To achieve it in BroadwaySQS just set wait_time_seconds to e.g. 10 seconds (or any value from 1 to 20) and receive_interval to 0 in producer’s options.

Example:

def start_link(_opts) do
  producer = Application.get_env(:scanner, :broadway_producer, BroadwaySQS.Producer)
  queue_url = Application.get_env(:ex_aws_sqs, :queue_url)

  Broadway.start_link(__MODULE__,
    name: __MODULE__,
    producers: [
      default: [
        module: {
          producer,
          queue_url: queue_url, wait_time_seconds: 20, receive_interval: 0
        },
        stages: 1
      ]
    ],
    processors: [
      default: [stages: 50]
    ]
  )
end

Mocking behaviour in child processes

Imagine you have a process which is responsible for consuming some sort of queue and spawning a separate process for each given job.

Each of the job processors later called workers communicates with some infrastructure layers for eg. external API, database, etc. We are injecting infrastructure to workers via configuration:

defp current_impl do
  Application.get_env(:scanner, :ensure_file, __MODULE__.AwsImpl)
end

Each injected implementation must implement complementary behaviour for eg.

defmodule Scanner.Scans.EnsureFile.Impl do
  @callback call(String.t, String.t) :: {:ok, any()} | {:error, any()}
end

In this setup, we could mock the whole behaviour using Mox :

# test_suport.ex or other file with your test configuration
Mox.defmock(Scanner.EnsureFileMock, for: Scanner.Scans.EnsureFile.Impl)
Application.put_env(
  :sanner,
  :ensure_file,
  Scanner.EnsureFileMock
)

# actual test
expect(Scanner.EnsureFileMock, :call, fn ^bucket, ^path -> 
  {:ok, :response} 
end

When we run this test with the given context we would gen an error:

(Mox.UnexpectedCallError) no expectation defined for Scanner.EnsureFileMock.call/2

The reason is that Mox by default works in private mode which is applying our mocked behaviour only for the current process - in our scenario our tested module spawns workers which are the real consumers of our mock and the implementation is only mocked for our test process. To allow our child process to consume the mock we need to set Mox in our test the global mode:

setup :set_mox_global

Or we could use:

setup :set_mox_from_context

Which will set the private mode only when a test has async: true otherwise it would use global mode.