The Open-Closed Principle at an architectural level

Introduction

This is the first article of a series about the SOLID Principles applied at an architectural level. If you are familiar with the SOLID Principles for class design in OOP and have ever wondered if you can use them when designing the architecture of a system, I will try to give you some insight.

At a class level, the Open-Closed Principle (OCP) states that a class is opened for extension but close to modification, that is, you should be able to extend a class behavior without modifying it. This is usually done by extending the class, either using inheritance or composition.

At an architectural level, we are not trying to modify the functionality of a piece of the system (a process, daemon, service or microservice, what best suits your architecture) but instead, add new pieces leveraging the work already done. In order not to modify an existing piece, your system needs to be fully decoupled. I’m going to focus on an event-driven system where any services communicate via a message queue. This could be ActiveMQ, RabbitMQ, ZeroMQ, Kafka or any other, but I’m going to use Kafka terminology of Topic, publisher and subscriber, and Kafka’s ability to have different subscribers for the same topic.

Messaging Systems

In Image 1 you can see the general use case: a publisher sends
messages (events) to a topic, then several subscribers can obtain the events back from the topic. The arrows show the communication flow. Assuming that the publisher and subscribers are microservices, the double rounded rectangles are indicating that there are several instances of that particular microservice. In this case, we have four microservices, Publisher, Subscriber 1, Subscriber 2 and Subscriber n, each one with multiple instances.

A specific example

Now let’s move on to a specific example. Imagine we work in a car rental company and are in charge of building a new vehicle availability system. A simplified view of the whole rental process could be the following:

During the car rental, a rental agreement is made and the customer picks up the car. The car availability diminishes by one. Then there is the time using the car (rental time) and finally, the return of the car, the car check-in. Car availability increases by one. In both cases, car rental and car check-in, we are saving a rental agreement to a database so we can define an event “RentalAgreementSaved” that will be fired after saving the data. This event will be stored in the topic RentalAgreementSaved. So for the moment, we have two publishers sending messages to a topic, the microservice CarRental and the microservice CarCheckin.

We now need to define the content of the message. As the topic intent is to reflect that a rental agreement has been saved, the minimum amount of information we need is the agreement ID. But the main purpose of the system is to track vehicle availability so it would be nice to have a Status field to help us. This field can have two possible values:

• Active. This means the car is being used by the customer.
• Closed. This means that the customer has returned the car and the check-in process has been done.

One possible option in JSON for the CarRental microservice is:

{"Status": "Active",
"RentalAgreementID": 1234
}

And for the CarCheckin microservice:

{"Status": "Closed",
"RentalAgreementID": 1234
}

The Status field could have been obtained from the database, loading the Rental Agreement by his ID, but if we only want to track the availability, it is easier and more performant to have it directly in the JSON message. We’ll talk about this later.

Once we have the publishers and the format of the message they send, we can complete the picture

The CarAvailability microservice will consume the messages sent to the topic “RentalAgreementSaved”, increasing the availability by one if Status is Closed and decreasing it by one if Status is Active.

We now have a working system fulfilling the objective, that is, calculating the car availability. Can we extend it to do other useful work? Can we really apply the OCP Principle?

Extending the system

Imagine that we want to generate an invoice for the customer once the rental process is finished. We have an Invoicing microservice and we can subscribe it to the “RentalAgreementsSaved” topic. When the Status is Closed, this service can obtain the data for the rental agreement from the database (the rental agreement ID is within the message) and the customer data from a Customers table, linked to RentalAgreements table. With all this information, Invoicing microservice can issue an invoice for the customer. The picture is as follows:

We have extended the functionality of our system, without modifying it, just leveraging the fact that multiple subscribers can subscribe to the same topic. So yes, the OCP Principle can be used at an architectural level!

The Law of Demeter

We are proud of our new abilities and want to add new functionality, sending an email to the customer thanking him or her for using our services. As we did with the Invoicing microservice, we can obtain the rental agreement from the database, and then the customer information from the Customer linked table. But this is not very efficient, and our CustomerThanking service has nothing to do with a rental agreement. In fact, this doesn’t follow the Law of Demeter, and we want good practices in all our system.

What we can do now is to modify the message content for the
“RentalAgreementsSaved” topic and add a field “CustomerID”. The JSON is as follows:

{"Status": "Closed",
"RentalAgreementID": 1234,
"CustomerID": 8965
}

But, wait, modify the message content? We are now breaking the OCP principle! It seems that, in the end, we have to give up.

Bounded Contexts

Well, there is still something we can do, and Domain Driven Design (DDD) comes to the rescue. If we have divided our domain in Bounded Contexts, we can take advantage of it: in a simplistic model for the system we are studying, we can identify the following Bounded Contexts:

• Rental agreements.
• Customers.
• Vehicles.
• Rental agent. The user of the system who is doing the rental agreement.
• Broker. Usually, customers don’t rent directly but through a broker.

All these entities appear within the rental agreement but are Bounded Contexts on their own. So, when first designing the message format, we can include the main Bounded Contexts related to the operation we are performing. In this case, the initial message content design could have been:

{"Status": "Closed",
"RentalAgreementID": 1234,
"CustomerID": 8965,
"VehicleID": 98263,
"RentalAgent": 24352,
"Broker": 6723
}

With this message for the first time, we can implement the CustomerThanking microservice without breaking the OCP Principle and complying with the Law of Demeter. Not to mention, we have the building blocks for new business needs that can appear later. Some quick examples:

• Calculating rental agents commissions.
• Economic information for a broker.
• Anything related to vehicle maintenance.
• …

The most important thing when designing the message content in this way is that we open the door to add a lot of new functionalities related to this event that could never have been devised initially, and without breaking anything already in place.

Events and message data

How is a message composed? What is the necessary data in a message?

To answer these questions, we first need to know the different message types we are dealing with and the purpose of them. Firstly, a message represents an event, which is a fact, something that has already happened. This is why we name our events in the past, they are things that have occurred and that we can’t change. As we are storing our events in a Topic, we give this Topic the name of the event. For a good understanding of events, I suggest this talk from Jonas Bonér.

But, what is the purpose of an event? I know two main types of events and the purpose of them is:

• Represent a fact.
• Build a stream of data

Events that represent facts are used in systems like the one we are describing. The main purpose is to communicate that something has happened and provide some useful data related to this fact. We try to provide just the information needed and nothing else, and a good heuristic can be to provide the ID of the Bounded Context entities related to the event.

Events that build a stream of data are used in Big Data systems, where you have a huge amount of information traversing the system and you apply several transformations to it. In this case, the event carries as much information as we can provide, we don’t want our system to look elsewhere to perform the transformation because this will incur in an extra cost.

Minimizing the message information

Why is it important to minimize the amount of information when we want to represent a fact? Let’s show an example.

Imagine we want to add new functionality to the system, a Recommendations microservice that will send an email to the customer with possible offers based on his or her profile. Let’s make it simple and suppose we just need the customer’s age to issue the recommendation. We decide that we don’t want to pay the extra cost of going to the database to obtain the age, and so we store it in the message (forget for a moment the OCP Principle, we are analyzing the impact of adding new data to the message).

{"Status": "Closed",
"RentalAgreementID": 5678,
"CustomerID": 8965,
"VehicleID": 98263,
"RentalAgent": 24352,
"Broker": 6723,
"CustomerAge": 27
}

The system picture is now as follows:

We are happy because we have a nice, decoupled system. But, is it really decoupled? Imagine we want to modify our recommendations algorithm to take into consideration the customer’s driver license issue date. It’s easy, we just need to add this field to the JSON. But in this case, our microservices are not really decoupled, every time we need a field, we have to change the subscriber, and the publisher! We’ll probably need to modify every publisher and every subscriber. Our microservices are tightly coupled, and a very ugly coupling because we don’t realize this until we need to change the system.

We can think that, if we add every possible field to the message data, everything will be fine and we won’t need to modify the publishers nor the subscribers. But systems evolve over time, and we’ll always have a new field added to our model and the need to modify every microservice. So this strategy won’t work.

The best thing we can do is to provide enough information inside the message to fulfill the original use case we considered in our initial design, but also to make it usable for new microservices we are not aware at this moment. A good starting point is to include the ID of the major Bounded Contexts entities involved or related to the fact we are communicating with this event. Of course, this will break the Law of Demeter, new microservices will need to traverse several entities, but this is a tradeoff we need to do. And that is what Software Architecture is all about, to make good tradeoffs in order to have the best possible system. The ability to follow the OCP Principle is something so important and useful that justifies breaking sometimes the Law of Demeter.

Summary

An event-driven system gives us a great opportunity to apply the Open Closed Principle at an architectural level, leveraging the work we have already done and extending it in unsuspected ways. However, we need to design carefully the content of the events and be aware of the possible coupling a bad design can introduce. The design should be guided by the intent of the system, a good design for one objective (for example, a stream of data in a Big Data system) can be a terrible design for another (an event-driven system reflecting facts). Domain Driven Design’s Bounded Contexts can provide us with some guidance about the content of an event. And architecture is about making decisions and tradeoffs, maximizing OCP probably means minimizing the Law of Demeter, so we need to be wary and find a balance.