Make Your Monolith Majestic
This post is a follow up to Why a Monolith might be better than a Microservice Architecture, where we talked about common misconceptions and the difficulties of implementing a solid Monolithic or Microservice architecture. In this post, I would like to go more in depth about the design philosophies, that I believe are important when building a great Monolith. Of course, we will also discuss ways to grow your application into Microservices.
A Quick Word on Monolith-First
Whenever we start working on a new brilliant idea or application, everything is very fragile and any resistance can be fatal. We don't want to abandon a good idea because it got too complex too soon. We also know that setbacks are inevitable, so let's make sure it isn't the technology that's holding us back.
We want to be able to build a Walking Skeleton, where we create the blueprint for all major architecture components and link them together. By doing this we create a shell that we can then use to build the most important functionality that delivers business value. The latter is also known as prototyping. The fastest way to do this, according to Martin Fowler among others, is with a Monolithic architecture, hence the name Monolith-First.
Through prototyping and working towards a Minimal Viable Product, we learn far more about what we are doing, compared to spending weeks on getting our infrastructure and pipelines ready for Microservices. We want to focus on building the core functionality of an application, striving for a fast time to market. This way we get user feedback early on, which is extremely important to gauge whether users are interested in your product. In other words, start with business value and focus on technical excellence later.
Design Pay-off Line
In one of his articles, Design Stamina Hypothesis, Martin Fowler discusses whether it is worth the effort to design our software well. This ties in directly to the common misconception about Monoliths, where we are often mistaken that a Monolith is a synonym for a Big Ball of Mud. Rather than this being an inescapable truth, I believe this has more to do with being conscious of when we need design and when we don't. When we neglect architecture and design for too long we are at risk of ending up with a Big Ball of Mud. Not making a conscious decision when we have to, is still a decision. It usually doesn't end up the way we want it to.
Little design up front will get you started faster, this is great for prototyping and trying new ideas. However, there will be a cut-off line, where the lack of design will start to slow you down significantly. After you break the breakeven-point of design or no-design, you will start to pay debt for every change to the application. For some applications, that don't regularly change, this might be acceptable. For most applications, however, the requirements are constantly changing and evolving.
When we are working on a Walking Skeleton or prototype, we ideally want to get started as fast as possible. As soon as we see that our prototype is viable - it has the potential to be successful - and we are moving forward to a MVP, it becomes essential that we take time to refactor our application to focus on good architecture and design. This will give us tremendous benefits in the future. And as I already stated before, when we neglect to introduce a good design when we need to, we're gonna have a bad time.
Building a Majestic Monolith
Be Honest about the Health of your Monolith
It is important that we are honest about the current health of our application. If we have a system that is based on no-design, that even turned into a Big Ball of Mud, we first need to make sure we focus on technical partitioning by introducing structure into our application. A good place to start is to bring the classes or code of the same kind together. Think about grouping all entities in one package, and all repositories in another, and maybe moving all SOAP APIs to one place. We can achieve this by effectively introducing a Tier-based Architecture or maybe even an Hexagonal Architecture into our application. Make no mistake, just doing this can be quite the challenge already.
Now that our application has a new structure, we can focus on exactly what our application is functionally doing. We can use a design concept, like Domain-Driven Design, to closely look at the core of what our application is doing. Once we have clearly defined this, we can start to functionally break up our application into one or more domains. In this process we are further structuring our code, not only on the technical level, but also on the functional level. This means that, for example, we don't just place all entities together, we also group entities based on what they do functionally (bounded contexts), allowing us to further increase the cohesion in our system. This process is also known as domain partitioning, and is an essential piece to later transition to a (Micro)service architecture. More on that later.
I would like to leave you with two questions you can ask your self and your team to help you get started (1) What was the hardest problem you solved? And (2) What assumptions did you make along the way. The first question will likely point to in the direction of what your core domain is. The second question will help you understand the boundaries and scope of your application. I originally read about these questions in The Software Architect Elevator, by Gregor Hohpe, and I hope they will help you as well.
Apply Conceptual Compression Everywhere
Conceptual compression is the key to building a large integrated system, such as a Monolith. It essentially means that we are creating useful abstractions from concepts in the application to make them easier to digest and use. This can be achieved by hiding the implementation details underneath a clearly defined interface or API. By doing this the developer using the API doesn't need to know what the API does internally and how it is structured. All the developer knows is how to interact with it, what goes in and what comes out.
A concrete example of conceptual compression can be found in Object Relation Mapping (ORM) frameworks, such as Hibernate in Java or Eloquent in PHP. An ORM allows the developer to use a clearly defined API, in a language we already know, to perform actions on a database. In other words, we have created a layer of abstraction, so the developer doesn't need to worry about all the details of writing good SQL queries.
Now this is also something we can employ in our own code. Let's say we use Domain-Driven Design (DDD) in our organization and we have defined multiple domains that we need to implement. We want to be able to have a clearly defined interface to interact with this domain. So according to DDD, we can build an aggregate. On this aggregate we primarily define the behavior that we can use to interact with the domain concepts. By doing this, another developer can easily hook into an existing domain, because the API fully abstracts all the complexities in that domain. We get the ability to just tell the domain what we want it to do. Powerful stuff, right?
By being aware of these abstractions and what they mean for us, we are more inclined to be aware of the principles of Loose Coupling, we can only access a component through its clearly defined API, and High Cohesion, the concepts that belong together are closely grouped. Another byproduct of conceptual compression is higher development velocity and overtime healthier application.
Not Everything Belongs to the Monolith
As our application grows and we learn more and more about our domain, we begin to realize that not everything we need to build belongs to our core domain. At some point in the lifespan of our application we are required to build new foundations that enables us to improve the customer focus or allows us to expand our application to a new platform.
Let's say we are currently running a large FinTech application, where we are streamlining the world of banking. We might get asked to create a real-time chat-bot for customer service, so a user of our application might ask questions and get suitable answers on-the-fly. We could try to cram this feature into our existing monolith, but we might run into some issues. The rest of our application is not geared towards real-time messaging the chat-bot requires. In this example the chat-bot has nothing to do with our domain of banking, it is customer service oriented instead. We want to optimize for the design characteristics the chat-bot requires, without changing the embodying these new characteristics in our Monolith.
It would make sense not to implement this chat-bot functionality inside of the Monolith. Instead, we could opt for a new stand-alone (micro)service, that is built and optimized for this specific feature. The (micro)service has its own lifecycle and can be changed without changing the Monolith. This allows us to keep our Monolith intact, while still providing new features that customers are requesting.
To summarize, if it is clear the functionality is not part of your core domain or does not fit in the design of our monolith, it is perfectly fine to position this new feature outside of the monolith as a stand-alone service.
The Modular Monolith
To take a small step back, in the last three sections we have talked about some elements that make a Monolith Majestic. We have talked about technical and domain partitioning to create a solid structure for our application, and we have focused on component-oriented design to allow for loose coupling and high cohesion. We also discussed how we can apply techniques such as Conceptual Compression to keep the Monolith understandable for the developers and architects. To clarify further, a Majestic Monolith seems to look a lot like a Modular Monolith.
If you want to read more about becoming a Modular Monolith, checkout Deconstructing The Monolith by Shopify. Shopify has a incredibly large Monolith built in Ruby on Rails. And they have been able to remain a Monolith for a very long time. The article I linked above takes you on their journey of creating their version of a Majestic Monolith.
Transitioning to Microservices
First make the change easy, then make the easy change. ~ Kent Beck
Kent Beck has spoken some very powerful words. In one way or another we all prefer to get to our end-goal as quickly as possible, but as we have already seen, in software development shortcuts might give you a lot of headaches in the future. When we move to a Microservice architecture too quickly, where either our organization or our software isn't ready, we are, undoubtedly, gonna have a bad time.
We have to follow the process of making our Monolith Majestic. A Microservice is very delicate and the architecture is only as successful as the quality of these services. It is essential that the Microservices are created based on the principles of domain partitioning, as this is the only correct way to slice an existing application.
If we fail to do this we will end up with Microservices with the wrong granularity, thus potentially distributing our Monolith. We often fall into the trap of making our Microservices actually micro. When we do this, we are at risk of introducing too many interdependencies on different Microservices. This results in a complex infrastructure where one Microservice isn't able to start and finish a procedure on its own, it needs many different services to accomplish this. In other words, our microservices are too fine-grained, resulting in low cohesion. Doing this results in a living nightmare, where you now need to worry about distributed and compensating transactions. These are actually very helpfull patterns, but if we choose to employ them and introduce their complexity to our system, we need to have a good business driven reason for that. If we don't, we probably need to reconsider our architecture.
A Microservice can actually be quite large, also known as coarse-grained. When we correctly partition our Monolith, we realize some parts of our domain can't be sliced up in too many little pieces. When the domain belongs to one and the same bounded context, we shouldn't even attempt to slice it. Instead, you might choose to transition the entire bounded context to a Microservice.
After going through the process of the Majestic Monolith, we might realize we don't need Microservices after all. A Monolith based on a good design goes a very long way. We could always choose create a hybrid solution, where our core domain is still a Monolith, but other bounded context are stand-alone Microservices that the monolith depends on.
Both Monoliths and Microservices, and even a combination of the two, has its place. They are all great architectures in the right context. The real challenge lies in knowing your own context. Only when you are deep in the mud, working on your problems, getting to know your context, can you make these decisions the right way.
Transforming your Monolith to a Majestic Monolith and extracting Microservices still doesn't guarantee a smooth transition. Technically, we probably won't face too many problems, because the hard work has already been done. However, the organizational aspect is probably even more important. If the organization and its development teams have not adopted the DevOps Way of Working, we might still face a lot of challenges before we can fully take advantage of our new and improved architecture.
In a future blog post we will take a look at the Majestic Monolith in a Serverless application. If you're interested, consider subscribing to Papers by Draftsman.
