In this episode we talk with Daniel and Steve about their experience with event-driven systems and shed some light on what they are and who they might be for. We explore topics like the complexity of setting up an event-driven system, the need to embrace eventual consistency, useful tools for building event-driven systems, and more.
Dropbox Engineering tells the tale of their new SOA:
The majority of software developers at Dropbox contribute to server-side backend code, and all server side development takes place in our server monorepo. We mostly use Python for our server-side product development, with more than 3 million lines of code belonging to our monolithic Python server.
It works, but we realized the monolith was also holding us back as we grew.
This is an excellent, deep re-telling of their goals, decisions, setbacks, and progress. Here’s the major takeaway, if you don’t have time for a #longread:
The single most important takeaway from this multi-year effort is that well-thought-out code composition, early in a project’s lifetime, is essential. Otherwise, technical debt and code complexity compounds very quickly.
SMTP should be blocked on public networks.
Email technology offers no effective means to stop phishing, so it’s been a runaway success for the attackers, and a disaster for millions of victims.
Sunsetting SMTP is clearly necessary and feasible. So, I’ve drafted a protocol called TMTP and I’d like to tell you about it.
What is a microservice, and what is a monolith? What differentiates them? When is a good time for your team to start considering the transition from monolith to microservice? And does using microservices mean you can’t use a monorepo?
This had me (literally) lol-ing and thinking about Kelsey Hightower. Viewer beware: there is an NSFW moment near the end. Put on some headphones if you have to, because it’s worth every effort.
Fascinating look behind the scenes at both the process of rewriting a massively used application and the particular architectural choices made along the way. The approach used was at once incremental and all-encompassing, rewriting a piece at a time into a gradually growing “modern” section of the application that utilized React and Redux. And the results? 50% reduction of memory use and 33% improvement in load time… not too shabby.
What’s the front-end equivalent of a micro-services architecture? A micro-frontends architecture of course. This approach makes a ton of sense, though in my opinion you will definitely want to have an internal components library and some cross-frontend coordination so your UI doesn’t degrade into a series of disconnected, disjointed experiences.
It’s hard to argue against the benefits stated by author Cam Jackson:
Micro frontends are all about slicing up big and scary things into smaller, more manageable pieces, and then being explicit about the dependencies between them. Our technology choices, our codebases, our teams, and our release processes should all be able to operate and evolve independently of each other, without excessive coordination.
This is Segment’s story from monorepo to microservies back to monorepo — “from 100s of problem children to 1 superstar child.”
Software Engineer Alexandra Noonan writes on the Segment Engineering blog:
As time went on, we added over 50 new destinations, and that meant 50 new repos. To ease the burden of developing and maintaining these codebases, we created shared libraries to make common transforms and functionality … Over time, the great benefit we once had of reduced customization between each destination codebase started to reverse. Eventually, all of them were using different versions of these shared libraries.
The woes of operational overhead with each expansion into more microservices.
The number of destinations continued to grow rapidly, with the team adding three destinations per month on average, which meant more repos, more queues, and more services. With our microservice architecture, our operational overhead increased linearly with each added destination. Therefore, we decided to take a step back and rethink the entire pipeline.
One of the original motivations for separating each destination codebase into its own repo was to isolate test failures. However, it turned out this was a false advantage. With destinations separated into their own repos, there was little motivation to clean up failing tests.
I’d love to dig into this story more on The Changelog with the team behind this transition back to a monolith and discuss the deeper details of their lessons learned.