The Problem
Inside a single @Transactional method, it is genuinely tempting to write to the database and then publish a RabbitMQ message describing what you just wrote. The code reads beautifully: one method, one unit of work, two side effects, atomic intent. The atomicity is an illusion.
The database and the broker do not share a transaction. A plain RabbitMQ publish sends the message to the broker as soon as that line runs; the database decides the outcome later, at commit, and a rollback cannot recall a message that has already left. So a publish inside a @Transactional method can go wrong in two directions, depending on where the failure lands relative to the publish:
- The phantom message. The publish goes out, then the commit fails. With JPA-style optimistic locking this is not exotic: the version check runs when the update is flushed, which is often at commit, after the publish line has already executed. The broker now holds a message about state that never existed, a consumer starts work the database says is not eligible yet, and if a retry later succeeds, the same message goes out a second time.
- The rollback-silenced message. The transaction rolls back before the message leaves, either because the failure comes before the publish line or because the publish was deferred until commit and is discarded with the rollback. Dropping that attempt’s message is correct. The bug is the retry: if it re-runs only the database write, which then succeeds on a fresh read, the message that write should have produced is never sent. The state is on disk and nobody was told. No exception, no log line, just a downstream consumer waiting for work that was never dispatched.
This is exactly the shape of a DAG engine. In the config-driven pipeline orchestrator I designed, a step completion merges its result into a JSONB column on the processing context under a @Transactional boundary, and the merged state decides which steps to dispatch next. When two sub-processors complete at almost the same instant, one merge loses the optimistic-lock race and rolls back. Early iterations of the engine published those dispatches from inside the transactional method. They never ran against real traffic: the orchestrator’s architectural review gate caught the pattern, and the rule below is what came out of that review.
The Approach
The rule: never publish a RabbitMQ message inside a @Transactional method. A message is something you earn the right to publish after the database has committed.
The cleanest shape I’ve found is to have the transactional method do its database work and return an intent, and let a non-transactional caller publish it. Put the two in separate beans so the transaction boundary is unambiguous, and put the retry loop in the caller so every attempt gets a fresh transaction and a fresh read:
@Singleton
public class StepCompletionHandler {
private static final int MAX_ATTEMPTS = 3;
private final StepMergeService mergeService;
private final DispatchPublisher publisher;
// constructor omitted
public void handle(StepCompletion completion) {
EvaluationResult result = mergeWithRetry(completion);
// Only reached once a transaction has committed.
for (DispatchIntent intent : result.dispatchIntents()) {
publisher.dispatch(intent);
}
}
private EvaluationResult mergeWithRetry(StepCompletion completion) {
for (int attempt = 1; ; attempt++) {
try {
return mergeService.mergeAndEvaluate(completion); // commits before returning
} catch (RuntimeException e) {
// Walk the cause chain: a conflict detected at commit usually arrives wrapped.
if (!isVersionConflict(e) || attempt == MAX_ATTEMPTS) {
throw e;
}
// Nothing was published; the next attempt re-reads in a new transaction.
}
}
}
}
@Singleton
public class StepMergeService {
// jakarta.transaction.Transactional: always its own transaction, so returning means committed.
@Transactional(Transactional.TxType.REQUIRES_NEW)
public EvaluationResult mergeAndEvaluate(StepCompletion completion) {
var context = contextRepo.findById(completion.contextId()).orElseThrow();
context.merge(completion);
return dagEvaluator.evaluate(context);
}
}The publish loop only ever sees an EvaluationResult from a transaction that committed. A version conflict, whether it surfaces at flush or at commit, throws out of mergeAndEvaluate before anything is published, and the loop retries against fresh state. The retry cannot drop the dispatch, because the publish is not inside the thing being retried, and it cannot duplicate one, because failed attempts never reach the publish. REQUIRES_NEW is what keeps that true if a caller is ever already inside a transaction: with the default propagation, the method would join the outer transaction, and “returned” would no longer mean “committed.”
Return-and-publish is not the only shape. An after-commit callback registered with the transaction manager gets the same ordering without changing the method’s signature, as long as the retry re-runs the whole transactional method that registers the callback. Both leave one narrower gap: a process that dies, or a publish that fails, after the commit. A transactional outbox closes both: write the dispatch intent to an outbox table inside the same transaction, and have a relay publish from that table after commit, at the cost of a table, a relay, and at-least-once delivery that consumers already need to tolerate.
Results
The rule sounds pedantic in code review and is invisible in normal testing. Both failure modes need a rollback at the wrong moment, and the likeliest source of one in this engine is two completions racing for the same context: precisely the case a single-threaded unit test never sets up. That is why a review gate caught it rather than a test. The orchestrator’s gate passed on its third attempt; the first two iterations missed this invariant (along with half-logic in terminal evaluation), and the gate sent them back before any traffic flowed. Moving the publish outside the transaction was a small refactor for the class of failure it removes.
Two adjacent rules earn their keep alongside this one. Publish should be idempotent: include a deduplication key in every message so consumers can drop replays (the orchestrator enforces a unique constraint on context, step, and attempt, so a redelivered dispatch can never double-execute a step). Publish failures should be observable: log and meter every send, and use broker-level publisher confirms, because a publish that does not throw is not the same as a publish that arrived.