Spring AMQP: Dead Letter Queue with RabbitMQ

Not every message can be processed successfully. A malformed payload or a violated business rule will fail no matter how many times it is consumed. In this tutorial, we will explore how to route such messages into a Dead Letter Queue (DLQ) using Spring AMQP and RabbitMQ, and verify the behaviour with integration tests powered by Testcontainers.

Background

When a consumer fails to process a message, RabbitMQ will, by default, requeue it. If the failure is permanent, the same message is delivered again and again, blocking other messages and wasting resources. This is commonly known as a poison message.

RabbitMQ solves this with Dead Letter Exchanges (DLX). A queue can be configured with an x-dead-letter-exchange argument. When a message in that queue is dead-lettered, RabbitMQ republishes it to the configured exchange instead of discarding it. A message is dead-lettered when:

  • It is rejected by the consumer with requeue=false.

  • It expires due to its TTL.

  • It is dropped because the queue exceeded its length limit.

  • It exceeds the delivery limit of a quorum queue.

The queue bound to the dead letter exchange is our Dead Letter Queue. Messages that end up there can be inspected, fixed, and replayed at a later time without blocking healthy traffic.

The Use Case

We will build an application that consumes orders from the orders queue. Processing an order can fail in two different ways:

  • Permanent failure: an order with a quantity of less than one is invalid. It will never succeed, no matter how many times it is processed.

  • Transient failure: the ordered product is temporarily unavailable. It may succeed if we try again a moment later.

Instead of looping forever, a failed order will be routed to the orders.dlq queue. We will explore two approaches to achieve this, compare them, and finally combine them so that each type of failure is handled appropriately.

Publisher ──► orders (exchange) ──► orders (queue) ──► OrderListener
                                         │
                                         │ rejected
                                         ▼
                                    orders.dlx (exchange) ──► orders.dlq (queue)

Implementation

The Record

We start by defining an Order record that will be published and consumed as JSON:

record Order(Long id, String product, int quantity) {
}

Queue Topology

The topology is declared in OrderQueueConfiguration. Spring AMQP’s RabbitAdmin, which is auto-configured by Spring Boot, declares every Exchange, Queue, and Binding bean against the broker on startup.

@Configuration(proxyBeanMethods = false)
class OrderQueueConfiguration {

    static final String ORDER_EXCHANGE = "orders";
    static final String ORDER_QUEUE = "orders";
    static final String ORDER_ROUTING_KEY = "orders";

    static final String DEAD_LETTER_EXCHANGE = "orders.dlx";
    static final String DEAD_LETTER_QUEUE = "orders.dlq";
    static final String DEAD_LETTER_ROUTING_KEY = "orders.dlq";

    @Bean
    DirectExchange orderExchange() {
        return new DirectExchange(ORDER_EXCHANGE);
    }

    @Bean
    Queue orderQueue() {
        return QueueBuilder.durable(ORDER_QUEUE)
                .deadLetterExchange(DEAD_LETTER_EXCHANGE)
                .deadLetterRoutingKey(DEAD_LETTER_ROUTING_KEY)
                .build();
    }

    @Bean
    Binding orderBinding() {
        return BindingBuilder.bind(orderQueue()).to(orderExchange()).with(ORDER_ROUTING_KEY);
    }

    @Bean
    DirectExchange deadLetterExchange() {
        return new DirectExchange(DEAD_LETTER_EXCHANGE);
    }

    @Bean
    Queue deadLetterQueue() {
        return QueueBuilder.durable(DEAD_LETTER_QUEUE).build();
    }

    @Bean
    Binding deadLetterBinding() {
        return BindingBuilder.bind(deadLetterQueue()).to(deadLetterExchange()).with(DEAD_LETTER_ROUTING_KEY);
    }

}

The key part is orderQueue(). deadLetterExchange and deadLetterRoutingKey set the x-dead-letter-exchange and x-dead-letter-routing-key arguments on the orders queue. Without an explicit routing key, RabbitMQ would reuse the message’s original routing key when republishing it to the dead letter exchange.

The dead letter exchange and queue are ordinary RabbitMQ resources. There is nothing special about them other than being referenced by the arguments of orders.

Queue arguments cannot be changed once a queue has been declared. Redeclaring an existing queue with different arguments fails with PRECONDITION_FAILED. For queues that already exist in production, consider configuring the dead letter exchange through a policy instead.

Message Conversion

Orders are exchanged as JSON. We declare a MessageConverter bean, which Spring Boot will apply to both RabbitTemplate and @RabbitListener containers:

@Bean
MessageConverter messageConverter(JsonMapper mapper) {
    // Only types from this package may be deserialized based on the __TypeId__ header
    return new JacksonJsonMessageConverter(mapper, Order.class.getPackageName());
}

JacksonJsonMessageConverter adds a TypeId header to outgoing messages. When the target type cannot be inferred, for example when reading from a queue with RabbitTemplate, the converter relies on this header to decide which class to instantiate. Restricting the trusted packages ensures that a crafted message cannot trick the application into deserializing arbitrary types.

The Listener

OrderListener consumes from the orders queue. We also keep track of processed orders and the number of attempts made for each order so that we can verify the behaviour later.

@Component
class OrderListener {

    private static final Set<String> AVAILABLE_PRODUCTS = Set.of("Spring Boot in Action", "Spring in Action");

    private final List<Order> processedOrders = Collections.synchronizedList(new ArrayList<>());
    private final Map<Long, Integer> attempts = new ConcurrentHashMap<>();

    @RabbitListener(queues = ORDER_QUEUE)
    void process(Order order) {
        attempts.merge(order.id(), 1, Integer::sum);

        if (order.quantity() < 1) {
            throw new AmqpRejectAndDontRequeueException("Order %d has an invalid quantity of %d".formatted(order.id(), order.quantity()));
        }

        if (!AVAILABLE_PRODUCTS.contains(order.product())) {
            throw new IllegalStateException("Product %s is temporarily unavailable".formatted(order.product()));
        }

        processedOrders.add(order);
    }

    List<Order> getProcessedOrders() {
        return List.copyOf(processedOrders);
    }

    int getAttempts(Long orderId) {
        return attempts.getOrDefault(orderId, 0);
    }

}

Notice that the listener throws two different exceptions. Each of them represents one of the approaches that we will explore next.

Routing Failed Messages to the Dead Letter Queue

By default, a message whose listener throws an exception is rejected with requeue=true. RabbitMQ puts it back on the orders queue and delivers it again, forever. In order for the message to be dead-lettered, it must be rejected with requeue=false. Spring AMQP offers two ways to achieve that.

Approach 1: Retry, Then Reject

The first approach is configuration driven. We let Spring retry the message a number of times, and once retries are exhausted, reject it without requeue. This is configured in application.properties:

spring.rabbitmq.listener.simple.default-requeue-rejected=false
spring.rabbitmq.listener.simple.retry.enabled=true
spring.rabbitmq.listener.simple.retry.max-retries=2
spring.rabbitmq.listener.simple.retry.initial-interval=500ms
spring.rabbitmq.listener.simple.retry.multiplier=2

With these properties:

  • A failed message is retried within the consumer up to two more times, waiting 500 milliseconds and then 1 second between attempts. This gives a total of three attempts.

  • Once retries are exhausted, Spring Boot’s default MessageRecoverer, RejectAndDontRequeueRecoverer, rejects the message with requeue=false. RabbitMQ then dead-letters it to orders.dlx, which routes it to orders.dlq.

  • default-requeue-rejected=false acts as a safety net. Should retries be disabled, a failed message is still rejected without requeue and dead-lettered, instead of being redelivered to the orders queue endlessly.

The listener remains unaware of RabbitMQ. It simply throws IllegalStateException when a product is unavailable.

Retries happen within the consumer and do not involve the broker. Therefore, the x-death header that RabbitMQ adds to a dead-lettered message will report a single rejection, regardless of the number of retries.

Approach 2: Throw AmqpRejectAndDontRequeueException

The second approach is code driven. When the listener throws AmqpRejectAndDontRequeueException, Spring AMQP rejects the message with requeue=false, regardless of the default-requeue-rejected setting. This is what our listener does for an order with an invalid quantity:

if (order.quantity() < 1) {
    throw new AmqpRejectAndDontRequeueException("Order %d has an invalid quantity of %d".formatted(order.id(), order.quantity()));
}

On its own, this approach requires no configuration at all. The message is dead-lettered after the first attempt.

Comparison

Both approaches end with the message in orders.dlq and require a similar amount of effort: a handful of properties for the first, a single throw statement for the second. However, they do not behave the same way:

Retry, Then Reject AmqpRejectAndDontRequeueException

Driven by

Configuration

Code

Attempts before dead-lettering

Configurable, three in our example

One

Suitable for

Transient failures, such as an unavailable downstream service

Permanent failures, such as invalid payloads or violated business rules

Handles unexpected exceptions

Yes, any exception is retried and then rejected

No, only the exceptions that we explicitly convert

Coupling

Listener is unaware of Spring AMQP

Listener depends on Spring AMQP’s exception

In other words, neither approach is a replacement for the other. Retrying an invalid order is a waste of time and resources, while dead-lettering an order on its first transient failure gives up too early.

Combining Both Approaches

The natural solution is to use both: retry transient failures and dead-letter permanent failures immediately. There is a catch, however. When retry is enabled, every exception is retried, including AmqpRejectAndDontRequeueException. Without further configuration, an invalid order would still be attempted three times.

We can exclude AmqpRejectAndDontRequeueException from being retried with a RabbitListenerRetrySettingsCustomizer, which is declared in OrderQueueConfiguration:

@Bean
RabbitListenerRetrySettingsCustomizer retrySettingsCustomizer() {
    // Permanent failures are rejected immediately instead of being retried
    return settings -> settings.setExceptionExcludes(List.of(AmqpRejectAndDontRequeueException.class));
}

Now, an order with an invalid quantity is dead-lettered on its first attempt, while an order for an unavailable product is attempted three times before it is dead-lettered.

Verification

We will verify the implementation with an actual RabbitMQ instance provided by Testcontainers.

RabbitMQ Container

TestcontainersConfiguration defines the container:

@TestConfiguration(proxyBeanMethods = false)
public class TestcontainersConfiguration {

    @Bean
    @ServiceConnection
    RabbitMQContainer rabbitMQContainer() {
        return new RabbitMQContainer(DockerImageName.parse("rabbitmq:latest"));
    }

}

@ServiceConnection instructs Spring Boot to connect to the container, so no connection properties need to be defined.

Integration Tests

OrderDeadLetterTests covers the happy path as well as both types of failure:

@Import(TestcontainersConfiguration.class)
@SpringBootTest
class OrderDeadLetterTests {

    @Autowired
    private RabbitTemplate rabbitTemplate;

    @Autowired
    private OrderListener listener;

    @Test
    @DisplayName("When a valid order is published Then it should be processed and not be routed to the dead letter queue")
    void valid() {
        var order = new Order(1L, "Spring Boot in Action", 1);

        rabbitTemplate.convertAndSend(ORDER_EXCHANGE, ORDER_ROUTING_KEY, order);

        await().atMost(Duration.ofSeconds(5)).untilAsserted(() ->
                assertThat(listener.getProcessedOrders()).contains(order)
        );

        assertThat(listener.getAttempts(order.id())).isOne();
        assertThat(rabbitTemplate.receive(DEAD_LETTER_QUEUE, Duration.ofSeconds(1).toMillis())).isNull();
    }

    @Test
    @DisplayName("When an order has an invalid quantity Then it should be routed to the dead letter queue without being retried")
    void permanentFailure() {
        var order = new Order(2L, "Spring Boot in Action", 0);

        rabbitTemplate.convertAndSend(ORDER_EXCHANGE, ORDER_ROUTING_KEY, order);

        assertDeadLettered(order);
        assertThat(listener.getAttempts(order.id())).isOne();
    }

    @Test
    @DisplayName("When an ordered product remains unavailable after all retries Then it should be routed to the dead letter queue")
    void transientFailure() {
        var order = new Order(3L, "Spring Data in Action", 1);

        rabbitTemplate.convertAndSend(ORDER_EXCHANGE, ORDER_ROUTING_KEY, order);

        assertDeadLettered(order);
        assertThat(listener.getAttempts(order.id())).isEqualTo(3);
    }

    private void assertDeadLettered(Order order) {
        var deadLetter = rabbitTemplate.receive(DEAD_LETTER_QUEUE, Duration.ofSeconds(10).toMillis());

        assertThat(deadLetter).isNotNull();
        assertThat(rabbitTemplate.getMessageConverter().fromMessage(deadLetter)).isEqualTo(order);

        assertThat(deadLetter.getMessageProperties().getXDeathHeader())
                .singleElement(MAP)
                .containsEntry("queue", ORDER_QUEUE)
                .containsEntry("reason", "rejected")
                .containsEntry("exchange", ORDER_EXCHANGE);

        assertThat(listener.getProcessedOrders()).doesNotContain(order);
    }

}

The first test verifies that a valid order is processed on the first attempt and nothing is routed to the dead letter queue.

The remaining tests verify that each failed order arrives in orders.dlq with:

  • The original order as its payload.

  • An x-death header, added by RabbitMQ, stating that the message was rejected from the orders queue, which it received through the orders exchange.

The number of attempts tells the two approaches apart. An order with an invalid quantity is attempted once, as AmqpRejectAndDontRequeueException bypasses retries. An order for an unavailable product is attempted three times before RejectAndDontRequeueRecoverer gives up on it.

Conclusion

A Dead Letter Queue keeps poison messages from blocking healthy traffic without losing them. With Spring AMQP, we declare the dead letter exchange on the queue and then decide how a failed message gets there. Retrying and then rejecting with RejectAndDontRequeueRecoverer suits transient failures and acts as a catch-all for unexpected ones. Throwing AmqpRejectAndDontRequeueException suits failures that we know will never succeed. Combining both, with the latter excluded from retries, gives each failure the treatment it deserves. Testcontainers allows us to verify the complete flow against a real broker.