"Yet, that kills some options like: ekhm, transactions. The challenge with transactions is that they don’t scale (that’s why MongoDB is WebScale). They don’t scale, as you’d need to open a transaction, do some freehand operations, and then commit or rollback. That means you need to keep a connection open during that time."
Quite a few systems support transactions that are not scoped to a connection.
Greg, thanks for following up. Indeed, I might oversimplify this scenario for the sake of explanation. It's of course possible to have transactions without an open connection. As long as the database has a write-ahead log (and almost all do), with logical records, you can just correlate changes by providing a transaction id/bookmark/tag/whatever, then commit the changes as the last record. AFAIK, that's what Aurora Serverless and Turso allow (they time out transactions after a set period). Do you have in mind any other popular databases that allow that?
In fact almost all do it even if they *appear* to be scoped to a connection. There is a reason that many do it ... what happens when you need to support a distributed transaction? It is not uncommon in such a situation that transactions can last for minutes or even ... days?! :-D Note: I am not saying transactions *should* last so long but it is a possibility you have to consider.
There are legitimate edge cases where it could be quite problematic to not support losing a connection in the middle of a transaction. Given the transaction has been being written to the log etc it would seem reasonable to be able to continue it given a temporary issue such as a dropped tcp connection / reconnect.
The way many handle this is returning an identifier of the transaction back to the client using it. Given this occurs if a connection drop etc were to happen that client can then commit when they reconnect :)
"Yet, that kills some options like: ekhm, transactions. The challenge with transactions is that they don’t scale (that’s why MongoDB is WebScale). They don’t scale, as you’d need to open a transaction, do some freehand operations, and then commit or rollback. That means you need to keep a connection open during that time."
Quite a few systems support transactions that are not scoped to a connection.
Greg, thanks for following up. Indeed, I might oversimplify this scenario for the sake of explanation. It's of course possible to have transactions without an open connection. As long as the database has a write-ahead log (and almost all do), with logical records, you can just correlate changes by providing a transaction id/bookmark/tag/whatever, then commit the changes as the last record. AFAIK, that's what Aurora Serverless and Turso allow (they time out transactions after a set period). Do you have in mind any other popular databases that allow that?
In fact almost all do it even if they *appear* to be scoped to a connection. There is a reason that many do it ... what happens when you need to support a distributed transaction? It is not uncommon in such a situation that transactions can last for minutes or even ... days?! :-D Note: I am not saying transactions *should* last so long but it is a possibility you have to consider.
There are legitimate edge cases where it could be quite problematic to not support losing a connection in the middle of a transaction. Given the transaction has been being written to the log etc it would seem reasonable to be able to continue it given a temporary issue such as a dropped tcp connection / reconnect.
The way many handle this is returning an identifier of the transaction back to the client using it. Given this occurs if a connection drop etc were to happen that client can then commit when they reconnect :)