Having full insights into your application

Originally posted on devblog.coolblue.nl.

Penalties. In a soccer game, they should be a done deal as it is a relatively simple thing. The ball is at a free range at 11 meters with just one goalkeeper in a very wide goal. Simple right?

If you’ve been asked to take a penalty with a blindfold on, it becomes increasingly harder. A group of friends have shown that perfectly in a YouTube movie where they attempt to take penalties blindfolded. It shows that taking penalties blindfolded is kind of challenging. Of course, the lack of visual feedback is a big part of the challenge, but if we look at the video we see a lot more happening.

A lot of assumptions are being made by the person taking the penalty. In some cases he assumes a ball is present, he assumes that he is lined up to the ball or he assumes there is a single keeper in the goal. Tons of assumptions are made and all are based on what they expect from the situation at hand. Some assumptions are made on their last visual of the situation or simply because that assumption suits the standard operation.

The same parallels can be made for how we as developers tend to run our applications. We put a lot of effort in the initial deployment of our applications. We make sure that we can connect to a database, that we have access to file system and all other dependencies you might have to successfully run your application. We also make the assumption that these dependencies will never change. Once we have access to the file system, we will always have it. Databases are always available, aren’t they?

Of course, we all know this is not the case. A firewall change disrupts the connection between the application and the database, an update to the OS changes the access rights to the logging directory. Any dependency an application is relying on for its operation has an ability to change and disrupt your service.

Within Coolblue we put a strong emphasis on monitoring. It is strongly embedded in our Tech Principles which are a set of technical guidelines we have in our IT department to guide us in the daily choices that we make.

The monitoring principle guides us towards a couple of requirements we have for every application that we run. First of all, we want to have monitoring on our all dependencies. An application should be able to tell us which dependencies are ok and which ones aren’t. This helps us to find the root cause for disruptions quickly and work towards automated recovery of disruptions. Also, we like to know when we have a disruption of our application in favor of customers telling us that we have issues.

To be able to monitor our dependencies from the application we are following the guidelines on health instances as defined in the Simple Standard Service Endpoints by Beamly.

In order to get the most out of a cloud environment, we also need to tell that environment what the state of our service is. We need to give insights into the workings of the service in general and every specific instance that is running the application. For example, in order to make autoscaling work insights in the health of the application and each instance is key. Based on the health information the cloud environment can determine if the instance is unhealthy and should be terminated or the entire application is under load and needs to be scaled up to be able to handle the load.

These are just a few examples of our implementation of the monitoring principle. In following blog posts will give more insights in how we actually achieve our insights.