The Problem
The orchestrator gave us a config-driven DAG, but the DAG is only as generic as the steps inside it. The legacy system had 535 source-specific processor files because every upstream source (the external system a notification arrives from) had its own notification format, its own deduplication rule, its own credential flow, and its own document store conventions. If the new platform shipped 535 source-specific workers, we had moved the problem, not solved it.
I also had a hard constraint: the workers had to share a Postgres instance with the legacy system in the shared dev environment, and would in production. Different schema, same database. Hibernate’s default behavior under hbm2ddl.auto: update would have happily reached across to the legacy tables and tried to “fix” them.
The Approach
I designed ten generic pipeline stages to replace the 535 source-specific processors, keyed by capability rather than source:
Each stage is a @RabbitListener consumer wrapping a pure utility class. The utility takes a typed input, does its job, and returns a typed output. The consumer wrapper handles message decoding, error envelope, and the completion publish back to the orchestrator. Source-specific behavior is configuration: regex patterns for extraction, hash algorithms per source (SHA-512 and SHA-256), credential flow types, and document storage targets all live in versioned config rows and version-controlled fixtures, not in code. The stages that face upstream systems directly go further than HTTP: the authenticate and fetch stages drive real browser automation (Playwright) for the sources that require it. And where a stage needs data another service owns, it asks over gRPC (an artifact ledger on the orchestrator, a separate record-data service) rather than reaching into foreign tables.
The workers run Micronaut 5 on Java 25, migrated alongside the orchestrator without pausing feature work.
Three details made the workers safe to deploy alongside the legacy system.
Session tokens never persist. The authenticate stage returns only the auth-result metadata to the orchestrator. The session cookie or bearer token rides in the dispatch message to the fetch stage and is discarded after use. The orchestrator’s JSONB column never holds credential material. The companion essay on replay idempotency covers why that decision matters for replays.
Workers are read-only at the database level. I locked Hibernate to hbm2ddl.auto: validate in every environment and restricted entity scanning to the worker package so Hibernate has no idea the legacy tables exist. Even a misconfigured pod cannot issue DDL against the shared database. The validate mode also doubles as a deploy-time check that the workers’ view of the schema is consistent with the migrations the orchestrator owns.
The orchestrator owns all JSONB writes. Workers never mutate the processing context. They emit a completion message; the orchestrator merges. Combined with the orchestrator’s @Version optimistic-lock retry, this gives us serializable-by-construction state evolution without distributed transactions.
Every stage understands a dry run. The dry-run flag rides in every dispatch message, retries and replays included, and the stages with durable effects honor it: the store stage quarantines uploads under a scratch prefix and writes no document record, the package stage reports the package it would have created instead of creating it, and record lookup never auto-creates a record. The lookup, deduplication, and fetch stages still run for real against the source, so a dry run exercises the exact code path production would, minus the writes.
Results
Ten generic stages cover the first source pattern end-to-end on the dev environment, and the first real dry run against a live source there returned a mismatch receipt that surfaced two parity defects (a package-hash delimiter and a record-identifier keying rule) before any production traffic; neither staging nor production takes a promotion until every in-scope source proves end-to-end. The legacy processors are an oracle to sample where a legacy record exists, never a specification to reproduce: the stages are judged on outcome equivalence, not on replicating what the old code did. Adding a source is still configuration, a DAG template plus per-source settings for whichever stages that source touches, but that configuration now comes out of a discovery walk of one real notification and is proven by a dry run before promotion. An architecture review gate caught a hbm2ddl.auto: update setting in the workers’ base config at the gate rather than in an incident, and the entity-scan restriction stopped a class of cross-tenant accidents that the legacy system had never been forced to think about.