Azure Service Bus is the message broker I’ve reached for most often in .NET systems: orders handed from an API to a worker, integrations with partners that go down at the worst moment, jobs that must happen exactly once from the business’s point of view. The SDK makes the happy path short. Sending is a line, receiving is a handler.

The hard part is everything else. What happens when the handler throws? When a message can never be processed? When the same order is sent twice? When two events for the same customer must not overtake each other? This article goes through those cases one by one. Every output below comes from running the code against the official Service Bus emulator in Docker, with Azure.Messaging.ServiceBus 7.21 on .NET 10.

Running it locally

The emulator is a pair of containers, the broker and a SQL Server it keeps its state in, plus a Config.json that declares the queues. Two queues are enough for everything here:

{ "Name": "orders", "Properties": {
    "LockDuration": "PT30S", "MaxDeliveryCount": 3,
    "RequiresDuplicateDetection": true, "DuplicateDetectionHistoryTimeWindow": "PT20S",
    "RequiresSession": false } },
{ "Name": "carts", "Properties": {
    "LockDuration": "PT30S", "MaxDeliveryCount": 3, "RequiresSession": true } }

The client connects with a connection string that ends in UseDevelopmentEmulator=true, and from then on it is the same SDK and the same code you’d run against Azure. That alone changes how you work: the failure cases below can live in integration tests instead of being discovered in production.

Peek-lock: a message is lent, not given

By default a receiver gets a message in peek-lock mode. The message stays in the queue, locked for the receiver for the queue’s lock duration, and the receiver then settles it:

  • Complete: done, remove it.
  • Abandon: give it back now, to be delivered again.
  • Dead-letter: move it aside, to the queue’s dead-letter sub-queue.
  • Or do nothing, and the lock simply expires; the message comes back.

Each delivery increments the message’s DeliveryCount. When it passes the queue’s MaxDeliveryCount, Service Bus stops trying and dead-letters the message itself. ServiceBusProcessor wires this up for you: with AutoCompleteMessages, a handler that returns completes the message, and a handler that throws abandons it.

Here is a handler that always fails, as if the payment service were down:

await using var processor = client.CreateProcessor("orders", new ServiceBusProcessorOptions
{
    AutoCompleteMessages = true,
    MaxConcurrentCalls = 1,
});
processor.ProcessMessageAsync += args =>
{
    attempts.Enqueue(args.Message.DeliveryCount);
    throw new InvalidOperationException("payment service down");
};
processor.ProcessErrorAsync += args => Task.CompletedTask;
await processor.StartProcessingAsync();

With MaxDeliveryCount at 3, the emulator delivered it three times and then moved it to the dead-letter queue, with the reason filled in:

delivery counts seen: 1, 2, 3
DLQ: order-B-2002 reason=MaxDeliveryCountExceeded
     desc=Message could not be consumed after 3 delivery attempts.

Two things to notice. First, the retries are immediate: an abandoned message goes straight back to the queue. If the payment service is down, three attempts back to back achieve nothing. Retry transient errors inside the handler with a backoff (with Polly, for example), and let Service Bus’s redelivery be the outer safety net. Second, MaxDeliveryCount is a business setting, not a technical one. The default is 10; whatever you choose, it is the number of times your handler may run for one message.

Poison messages: don’t waste the retries

Some messages will never succeed: a payload that doesn’t parse, an order for a product that no longer exists. Retrying them only delays the inevitable and holds up the queue. The handler knows the difference, so it should dead-letter them at once, with a reason a person can read later:

var message = await receiver.ReceiveMessageAsync();
try
{
    var order = JsonSerializer.Deserialize<Order>(message.Body.ToString());
    // … process it …
    await receiver.CompleteMessageAsync(message);
}
catch (JsonException ex)
{
    await receiver.DeadLetterMessageAsync(message,
        deadLetterReason: "InvalidPayload",
        deadLetterErrorDescription: ex.Message);
}

A message with the body {not json went to the dead-letter queue on its first delivery:

DLQ: bad-1 reason=InvalidPayload deliveries=1
     desc='n' is an invalid start of a property name. Expected a '"'. …

The rule of thumb: transient errors throw (and get retried); permanent errors dead-letter (and get looked at). A handler that catches everything and completes the message hides bugs; one that throws on everything wastes the retries on messages that can’t recover.

Advertisement

Duplicates: at-least-once is a promise you have to keep

Service Bus delivers at least once. A sender that times out and retries may have sent the message twice; a handler that finishes its work but loses its lock before completing will see the message again. Duplicate detection handles the first case: on a queue with RequiresDuplicateDetection, messages with a MessageId already seen within the detection window are accepted and silently dropped.

for (var i = 0; i < 3; i++)
    await sender.SendMessageAsync(new ServiceBusMessage(BinaryData.FromObjectAsJson(order))
    {
        MessageId = "order-A-1001",
        ContentType = "application/json",
        Subject = "OrderPlaced",
    });
sent 3 with the same MessageId, queue holds 1

The key is choosing a MessageId that means something: the order’s id, not a new GUID per send. A random id defeats the whole feature.

Duplicate detection does not cover the second case, a redelivery of the same message after a lost lock, so the handler must still be idempotent: record the ids it has processed, or make the work itself safe to repeat (an upsert instead of an insert, a payment with an idempotency key). Duplicate detection is a filter; idempotency is the guarantee.

The dead-letter queue is a to-do list

Dead-lettered messages don’t expire on their own and don’t alert anyone. A dead-letter queue nobody watches is a slow data loss. Two habits help: alert on its length (Azure Monitor exposes the dead-letter message count per entity), and have a way to replay messages once the cause is fixed.

Replaying is receiving from the dead-letter sub-queue and sending a copy back. new ServiceBusMessage(received) copies the body and properties, which is convenient. But I found a trap when I ran it: on a queue with duplicate detection, the copy keeps the original MessageId, and the replay was silently dropped as a duplicate:

same MessageId replay arrived: False
replayed: replay-2d9a12:replay-1 subject=OrderPlaced body={"OrderId":"C-3003"}
          props=ReplayedFrom=PaymentServiceDown deliveries=1

So the replay tool gives the copy a new id derived from the old one, and records where it came from:

await using var dlq = client.CreateReceiver("orders",
    new ServiceBusReceiverOptions { SubQueue = SubQueue.DeadLetter });

foreach (var dead in await dlq.ReceiveMessagesAsync(maxMessages: 50))
{
    var copy = new ServiceBusMessage(dead) { MessageId = $"{dead.MessageId}:replay-1" };
    copy.ApplicationProperties["ReplayedFrom"] = dead.DeadLetterReason;
    await sender.SendMessageAsync(copy);
    await dlq.CompleteMessageAsync(dead);
}

Since the handler is idempotent on the business id inside the body, a replay of an order that was in fact processed does no harm.

Order: sessions, not hope

A plain queue with several consumers processes messages in parallel, so events for the same customer can overtake each other: a “checkout” handled before the “add milk” that came first. When order matters, sessions give it to you per key. Set SessionId on each message, enable sessions on the queue, and a session processor holds one session at a time per consumer, delivering its messages in order:

foreach (var (cart, change) in events)
    batch.TryAddMessage(new ServiceBusMessage(change) { SessionId = cart });
await sender.SendMessagesAsync(batch);

await using var processor = client.CreateSessionProcessor("carts", new ServiceBusSessionProcessorOptions
{
    MaxConcurrentSessions = 2,
    MaxConcurrentCallsPerSession = 1,
});

Six events for two carts, sent interleaved, came out in order within each cart, while the two carts were processed in parallel:

cart-1: add apple → add milk → checkout
cart-2: add bread → remove bread → checkout

Sessions also carry state: SetSessionStateAsync stores a small blob with the session on the broker, which survives a consumer restart. A handler that counted each cart’s events left 3 in cart-1’s state. That’s useful for a running total or the step a workflow has reached, without a database round trip.

The cost is throughput per key: one session is processed by one consumer at a time. Pick a session key that is as narrow as the ordering you actually need, a cart or an order rather than a whole tenant.

Scheduling: messages from the future

ScheduleMessageAsync enqueues a message that stays invisible until a given time, and returns a sequence number to cancel it with:

var sequence = await sender.ScheduleMessageAsync(
    new ServiceBusMessage("reminder") { MessageId = "remind-1" },
    DateTimeOffset.UtcNow.AddSeconds(5));
scheduled seq=11; after 2 s: nothing
later: remind-1 at +0.1s past schedule

It’s a clean way to do “remind the customer in 24 hours if the cart is still open” or “retry this integration in 10 minutes” without a scheduler of your own. Cancel with CancelScheduledMessageAsync(sequence) when the reason goes away.

Long handlers and the lock

If a handler takes longer than the lock duration, the lock expires, the message is delivered again to another consumer, and the work runs twice. The processor renews the lock automatically while the handler runs, up to MaxAutoLockRenewalDuration (five minutes by default). Set it to the longest a handler can reasonably take, and keep handlers short; work that takes many minutes is better split into steps.

A checklist I use

  • Choose MaxDeliveryCount on purpose, and retry transient errors with a backoff inside the handler.
  • Dead-letter permanent errors at once, with a reason and a description.
  • Give messages meaningful ids and turn on duplicate detection; still make every handler idempotent.
  • Watch the dead-letter queue and have a replay tool, with new message ids.
  • Use sessions when order matters, keyed as narrowly as possible.
  • Test the failure paths against the emulator, not just the happy path.

None of this is exotic. It’s the difference between a queue that moves messages and a system that keeps its promises when something goes wrong.