Architecture Weekly #173 - 1st April 2024
Architecture Weekly #173 - 1st April 2024
Welcome to the new week!
Let’s start with a big bang, at least for me. I'm no longer a Marten maintainer. As you know, sometimes in the project's lifetime, there's a moment when either you need to settle things or make hard decisions. Unfortunately, the latter occurred here.
I'm not happy that this is ending not as I hoped. Marten was a big part of my life: over 6 years as a contributor and over 5 years as a co-maintainer. Yet, the current collaboration model has ended, and the new didn’t emerge.
I'm not leaving the Event Sourcing and Event-Driven space. I plan to continue what I was doing, so if you need any help, feel free to reach out to me. The closest goals are the workshops on Techorama and DDD Europe. Then, I'll think about what's next and how much I need to adjust my course.
Read more:
Did you feel that you need to debug your mind? Did your (or your colleague's) biases impact your decision-making? How to improve it? How do you reason about the reasoning?
Laïla Bougriâ shared with us practical thoughts and the results of their research backed by the history of her projects. She gave us the talk she gave as the keynote at NDC London 2024 as our webinar. You can watch the recording now.
This talk gives good mental tools to reason, investigate, and do the sanity check BEFORE, which is a basis for evolutionary architecture and good enough design.
As always, after the talk, we had a nice Q&A during which Laïla shared additional heuristics. I’m really happy that Laïla agreed to join us and share her experience with us!
Check also Diana Montalion's talk from this month’s Explore DDD conference. In a similar spirit to Laïla’s webinar, she nicely explained how systems thinking can help analyse and understand what’s happening in our projects. She underlined the importance of understanding what’s behind our daily struggles, noticing patterns, and how analysing structures and building mental models can help us make better decisions.
CNCF announced a new database: Valkey! Well, almost, what’s Valkey? It’s a fork of Redis triggered by the change in their licence we discussed last week.
Valkey will continue development on Redis 7.2.4 and will keep the project available for use and distribution under the open source Berkeley Software Distribution (BSD) 3-clause license. (…)
Industry participants, including Amazon Web Services (AWS), Google Cloud, Oracle, Ericsson, and Snap Inc. are supporting Valkey. They are focused on making contributions that support the long-term health and viability of the project so that everyone can benefit from it.
Did you notice Microsoft missing? Yup, probably because they just released Garnet, their own Redis clone.
All that sounds splendid and backed by higher values, but frankly, if they were doing it because of that, they’d support the Redis creators and contribute back, making their development sustainable instead of benefiting from their work.
Still, all of that doesn’t look back from Redis. It sounds like they didn’t have a proper plan for what was next. I think the only solution for databases to survive such scenarios is to have managed service providing a high value to their customers. It also requires early investment in your own user base and community to be built around your product instead of just outsourcing it to the cloud providers.
We see more and more cases where the original tooling that became popular is getting some bottlenecks or trying to be used in scenarios where it wasn’t initially envisioned. For instance, databases were originally designed to be run on the bare-bone server with a dedicated disk, etc. That’s still a valid and probably the most performant usage, but in today’s industry, that’s not enough. Sometimes, we want to optimise for cost or other things. That made AWS create AuroraDB and Google AloyDB, so it used the Postgres engine and replaced the storage driver to store the files on cheap blob storage like S3.
Confluent is investing most of its effort in making Kafka Cloud-native and optimised for the Cloud. It’s not easy to change tools that were built traditionally. They created Kora, their cloud-only Kafka engine. They also have Apache Iceberg, the open table format for analytic datasets. Now, they came up with Tablefow.
They observed that even though messages piped through Kafka can have any schema, they typically have structure and share a similar schema, especially since they pushed to use their Avro Schema Registry. Given that and the need for cheaper, easier-to-replicate, and process storage, they thought, "Why not unify stream processing and batch processing?”.
So they did. It’s not a revolutionary idea, but it’s also an intriguing case study of how pure tech solutions can shift their approach once they understand their domain, how users use it, and their needs. It also shows (with pushed by Confluent Apache Flink) that SQL is not dying but doing well.
If you’re not yet on board with Kafka, it’s worth understanding it; the good starting point is a nice walkthrough Claudio Gargiulo:
If you’d like to learn more about innovative databases, check DuckDB; it can also use Apache Parquet files (with columnar storage format) just like Confluent’s Tableflow.
Read more about how Spanner is dealing with the CAP Theorem. It’s an interesting read to understand the guarantees they provide:
If you’re putting your bets on RedHat, you should reconsider their bets; it sounds like they started a partnership with McKinsey.
I’m interested in knowing if they want to use practices from McKinsey’s report on developer productivity. We should buy some popcorn for the next news around it.
Ok, truly, this was a joking introduction for yet another try at measuring developer productivity. This time on Martin Fowler’s blog by Abi Noda and Tim Cochran:
They proposed “qualitative metrics”:
Attitudinal metrics capture subjective feelings, opinions, or attitudes toward a specific subject. An example of an attitudinal measure would be the numeric value captured in response to the question: “How satisfied are you with your IDE, on a scale of 1-10?”.
Behavioral metrics capture objective facts or events pertaining to an individuals’ work experiences. An example of a behavioral measure would be the quantity captured in response to the question: “How long does it take for you to deploy a change to production?”
And added benefits that the benefit of using them are:
Qualitative metrics allow you to measure things that are otherwise unmeasurable.
Qualitative metrics provide missing visibility across teams and systems.
Qualitative metrics provide context for quantitative data.
The article is long and worth reading, but I think it won’t give you actionable metrics, but it should give you inspiration for working with your teams. I think that we just should stop calling those metrics. It’s worth analysing trends and trying to understand what it means in your organisation's context to be a good engineer. Trying to measure something that, by definition, is claimed as subjective and unmeasurable will still be a gut feeling. And gut feeling of experienced people can be a better indicator than dumb metrics, but it’s still gut feeling, so why cheat ourselves by calling that measurement?
Last but not least, I’d like to give big KUDOS to the tool that I’ve been a happy user of in the last few months: Lazy Git. I’m using Linux on my developer workstation for over a year, but I’m not a console-first user. I still like the desktop; I don’t want to cast the invocation or a spell; I just want to get stuff done. I was using TortoiseGit and Git Kraken, both nice tools, but I didn’t feel they were my ultimate choice.
Git is the base tool nowadays for developers, but it’s not user-friendly. It’s easy to do wrong rebase and break your work. I was really scared to do interactive rebases in the console or even resolve merge conflicts without the UI tool. But since I tried Lazy Git, I’m not scared anymore. Together with a plugin for VSCode, it helped me build a better developer flow while working on my Node.js projects like Emmett.
Try it, really, if I didn’t persuade you, check this article:
Check also other links!
Cheers
Oskar
p.s. I invite you to join the paid version of Architecture Weekly. It already contains the exclusive Discord channel for subscribers (and my GitHub sponsors), monthly webinars, etc. It is a vibrant space for knowledge sharing. Don’t wait to be a part of it!
p.s.2. Ukraine is still under brutal Russian invasion. A lot of Ukrainian people are hurt, without shelter and need help. You can help in various ways, for instance, directly helping refugees, spreading awareness, and putting pressure on your local government or companies. You can also support Ukraine by donating, e.g. to the Ukraine humanitarian organisation, Ambulances for Ukraine or Red Cross.
Architecture
📺 Diana Montalion - Systems Thinking for Software Professionals
Woody Zuill, Kevin Meadows - But, We Need Proof Before We Try It
Kurt Bittner, Pierre Pureur - Agile Architecture, Lean Architecture, or Both?
Claudio Gargiulo - Consuming a Kafka Topic Is Easy, Isn’t It?
Databases
CNCF - Linux Foundation Launches Open Source Valkey Community
Hannes Mühleisen - 42.parquet – A Zip Bomb for the Big Data Age
Testing
AI
Java
JavaScript
.NET
Stephen Toub, Scott Hanselman - Writing async/await from scratch in C#
AWS - Introducing the AWS Message Processing Framework for .NET (Preview)