Legacy batch jobs to SQL pipelines

Context

A General Ledger organization still depended on a large COBOL batch estate. Teams needed faster change cycles, better observability, and pipelines that could survive retries without corrupting accounting state.

Constraints

• Financial correctness beats migration speed; partial automation is worse than slow automation with proofs.
• Operators still think in batch windows, logs, and runbooks — new tooling must meet them where they work.
• No shortcut that hides which environment or ledger slice a job touches.

Approach

1. Inventory the real batch graph — which jobs are pure transform, which encode business rules, which are safe to parallelize.
2. Move incrementally to SQL-based ETL — roughly four-fifths of targeted legacy jobs over the modernization push, with explicit logging and idempotency at load boundaries.
3. Instrument for operators — log parsing and replay-friendly steps so “run it again” is a controlled action, not a crisis.
4. Pair with internal tooling — metadata-driven parameter surfaces so report owners stop editing production by email thread.

Trade-offs

Not every job belonged in wave one. Some stayed on legacy paths until test harnesses and sign-off rituals caught up. The win is a repeatable migration pattern, not a single big-bang rewrite slide.

Outcomes

• Material reduction in process time on modernized paths (on the order of ~95% improvement where measurement was tracked).
• Parameter and report management moved toward versioned, auditable workflows instead of ad hoc edits.
• Personal takeaway: reliability instincts formed here — EOY simulations, tier-2 support, hundreds of cross-team incidents — are the same instincts I bring to platform and integration work later.