|
Case Study
|
|
Implementing Durable Workflows on Postgres Without an External Orchestrator
|
Raman Varma explores how PostgreSQL can provide durable workflow execution without introducing a dedicated orchestrator such as Temporal or AWS Step Functions. Drawing on the architecture of Kestrel Workflows, Varma shows how familiar database primitives can handle the core requirements of workflow orchestration: SELECT ... FOR UPDATE SKIP LOCKED provides concurrent work claiming, primary key constraints enforce idempotent step checkpoints, and leases with a periodic sweeper enable recovery when workers fail.
The approach also supports long-running sleeps and human approval steps by persisting waits as database state, allowing workflows to survive application restarts or Kubernetes rescheduling. Because executions and checkpoints remain relational data, operational visibility becomes straightforward SQL rather than requiring the export of workflow data into a separate analytics system.
The architecture is not universally applicable. Varma highlights connection pressure, table churn, and dispatch latency as considerations at larger scale, while suggesting that a dedicated engine such as Temporal is preferable for workloads requiring thousands of workers or extremely low dispatch latency. For I/O-heavy automation with modest concurrency, however, the article argues that reusing an existing highly available PostgreSQL deployment can reduce infrastructure, security, and operational complexity.
This content is a short summary of a recent InfoQ article by Raman Varma, "Implementing Durable Workflows on Postgres Without an External Orchestrator".
To get notifications when InfoQ publishes content
on these topics, follow "AI, ML & Data Engineering", "Big Data", and "Database" on InfoQ.
|
|
Architecture Decisions Across Platforms, AI, and Delivery
|
Where should a platform standardize, and where should teams retain flexibility? What should an AI agent be allowed to access? What catches a coding agent's mistakes before its changes reach CI?
InfoQ's live online certification programs give you five weeks to work through decisions like these using QCon frameworks, facilitator context, and perspectives from experienced engineers at other companies.
Upcoming programs include:
- Architecture with Luca Mezzalira
Platform engineering trade-offs, decentralized decision-making, resilience, and AI architecture. Starts October 9.
- AI Security & Privacy Engineering with Katharine Jarmul
Sensitive-data flows, threat modeling, agent permissions, controls, evaluations, and governance. Starts October 14.
- AI-Assisted Engineering with Zichuan Xiong and Premanand Chandrasekara
Agent permissions, independent verification, sensors, and CI for unattended agents. Starts October 19.
- High-Performing Teams with Olimpiu Pop
Team structure, delivery flow, AI roles, engineering metrics, and building a target operating model. Starts November 17.
Explore the October cohorts.
|
Missed a newsletter? You can find all of the
previous issues
on InfoQ.
|
|
Industrial IoT systems can move from a fast pilot to a data-scale problem surprisingly quickly. As sensor counts and historical data grow, ingest, storage, and query workloads can expose architectural limits that weren't visible at proof-of-concept scale. This guide explores how to define a PostgreSQL performance envelope for IIoT workloads, anticipate capacity and storage constraints, and understand the tradeoffs of different scaling approaches. Learn how to evaluate database architecture for production-scale IIoT before performance limits become an operational problem.
Download the guide “The IIoT PostgreSQL Performance Envelope,” sponsored by Tiger Data
|
|
|
|
