How do you deploy new code continuously into a system that needs to support stateful, uninterruptible computations?
As web developers, we're used to valuing stateless computation, for good reason – reducing state makes everything easier. Servers (or cloud instances) hold no essential state, delegating any important data to databases like Postgres and Redis. This is a great architecture. Among other benefits, it makes zero-downtime deployments a breeze: to deploy a new code version, you simply spin up a set of new instances with the new git sha, add them to the load balancer, and kill the old instances after draining them of traffic (perhaps after 60 s).
However, some workloads don't fit this "short-duration request & response" mold. As an example, consider connecting OpenAI Realtime to Twilio's API, which requires you to: