Architecture Weekly #129 - 29th May 2023
Welcome to the new week!
Today, at 6 PM CET (UTC+2), we’ll have the 10th webinar for the Architecture Weekly paid subscribers community.
I’ll show Postgres Superpowers in Practice.
If you liked what I explained in my article and want to see that live come on in, there’s still some time. If you’re reading that later, that’s not an issue; you’ll get access to the recording.
Privacy is a boring thing for us developers. We don’t want to care about the data quality but about the technical aspects of it, like database type, query performance etc. Those aspects are also important, but because we didn’t want to apply proper data governance practices consistently, someone had to do it: the regulators.
Now we must decide whether we are baddies or buddies to our users. I talked about that during my talk at the Devoxx Greece conference. Watch it:
Want more proof? There you have it:
The Verge - Online age verification is coming, and privacy is on the chopping block
CNN - Meta slapped with record $1.3 billion EU fine over data privacy
BBC News - ChatGPT-maker U-turns on threat to leave EU over AI law
Vice - Debt Collectors Want To Use AI Chatbots To Hustle People For Money
I also wrote about another anti-pattern in data modelling: I'll just add one more field approach.
Programming origins are in mathematics. Scientists like John Von Neumann and Alan Turing built the foundations for today’s computers. That may be a reason why we tend to believe that 1 plus 1 will always equal 2. And that equation is not always true for event modelling.
The new property into event data may not add more information but degrade it. We may lose a precious business context. Why? Read more in my latest article:
If you’re looking for other resources about fixing mistakes in the event-driven world, then check:
The first is a general introduction to what can go wrong and which patterns we can use. The second does an excellent case study of the failure and how to do detective work, and also how Open Telemetry can help us with that.
And failure can be super costly, five mln dollars? Doable. That’s how much the outage cost Datadog. One of the parts of doing the right due diligence is to be transparent. Datadog didn’t want to be like that, then Gergely Orosz did great coverage. One of the lessons learned should be for all of us that if we care about our reputation and our customers, then we should be first and thorough to do that.
Confluent released Kora: a Kafka runtime dedicated to cloud-native usage. They wrote an interesting and frank note:
We realized that to make our vision a reality we had to build something designed for these constraints.
The article, besides the marketing fluff, is a good explanation of the various factors we should include in designing the cloud-native system. Significant differences exist between distributing to clients and running your own platform. That includes different needs for SLA, SLO, DevOps practices, scaling strategies, and operating team requirements. Read more in:
Speaking of the cloud-native approach, StackOverflow wrote an intriguing article discussing whether Kubernetes surprisingly didn’t make running on-premise easier and revert some cloud trends.
It’s undeniable that Kubernetes is not as easy to run on the scale as pictured. It’s still too low level for most developers (maybe because of their I just want to run a container... attitude). Still, it’s already serving as a backup plan in regulated industries like finances, where regulators are forcing to have alternatives if the geopolitics changes the cloud accessibility.
If we have a team already skilled in running our software, and our load is predictable without big spikes or the need for multi-region access, then it may appear that running on-premise may be much cheaper.
I don’t think we will revert to the cloud, as it gives us too many benefits. Yet, for bigger companies, on-premise still may be a good option, and it’s good that we’re embracing that.
I only hope we won’t be going into a hybrid world. It’s a madness that costs too much and skyrockets in complexity.
Interestingly the word Architecture in the context of the software industry is in the complexity context. Herbert A. Simon did that in 1962 in his work:
So it seems that tackling complexity is something that we’re still consistent and are not winning.
Check also other links!
Cheers
Oskar
p.s. I invite you to join the paid version of Architecture Weekly. It already contains the exclusive Discord channel for subscribers (and my GitHub sponsors), monthly webinars, etc. It is a vibrant space for knowledge sharing. Don’t wait to be a part of it!
p.s.2. Ukraine is still under brutal Russian invasion. A lot of Ukrainian people are hurt, without shelter and need help. You can help in various ways, for instance, directly helping refugees, spreading awareness, and putting pressure on your local government or companies. You can also support Ukraine by donating, e.g. to the Ukraine humanitarian organisation, Ambulances for Ukraine or Red Cross.
Architecture
Oskar Dudycz - How to deal with privacy and GDPR in Event-Sourced systems
Oskar Dudycz - Anti-patterns in event modelling - I'll just add one more field
StackOverflow Blog - Are clouds having their on-prem moment?
Netflix - ABAC on SpiceDB: Enabling Netflix’s Complex Identity Types
Gergely Orosz - Inside Datadog’s $5M Outage (Real-World Engineering Challenges #8)
Laila Bougria - Message processing failed! But what's the root cause?
DevOps
Frontend
Testing
AI
Santa Fe Institute - Workshop on AI and the Barrier of Meaning
Vice - Debt Collectors Want To Use AI Chatbots To Hustle People For Money