Giving an estimation on “how long it will take” is a frequent question you get asked as a software engineer or architect. In this post, I experiment with the c4 model to help me establish a conversation with my stakeholders and fellow engineers to identify drivers of effort, highlight uncertainty, and uncover unknowns that influence my answer to “how long will it take”. Having a conversation fosters trust and opens the door to previously unknown opportunities. In the course of discovery and discussion, you may find that there is a safer, shorter, cheaper, or more direct way of achieving the stated goal than previously envisioned.

TL;DR

• Using c4 diagrams help you have a structured process looking at a (static) software system possibly uncovering unknowns
• Adding different facets to the diagrams helps you look at the software with different lenses that all play into the estimation of effort: existing complexity, about-to-be-added complexity, your familiarity with code and technologies, missing answers to known questions, missing answers to unknown questions, refactoring work needed, testing strategy, etc.
• Different levels of abstraction make the c4 model ideal to engage in a structured well-informed discussion with your stakeholders as well as your fellow engineers.

The feature request

Imagine you recently joined a company building software for advert management as a software engineer. The product owner asks you how long it will take to implement the following “email feature”:

As a customer I want to receive an email after finalizing an order with a summary of the purchase and a reference number in order to be confident that the order took place and have a reference for later communication.

The feature request seems reasonable and the value is apparent as this is an expected behavior for an online shop selling adverts. You are about to say three days but bite your tongue and ask your product owner if she is ok when you give her an estimate the next day. She agrees and you leave the meeting excited ready to start working on your first feature!

diagrams to the rescue

Thankfully during onboarding another engineer told me about existing documentation in the wiki. So you start gathering more information about the software system.

system context diagram

After a short while browsing the wiki, you find the following system context diagram giving you a rough overview of involved personas and other external systems.

After looking at the (small) big picture you realize that the back office person might also benefit from receiving an email about a purchase. You start creating a discussion log to discuss with your PO. You also add a question about the company’s email server credentials to the discussion log as well as what sender should be used.

With great expectations, you click on the container diagram link in the wiki only to find an empty page. :-/

Container diagram

Time to dig into the project and find out what containers comprise the advertdesk software suite. After a while poking around you find out that there is a github action CI/CD pipeline that builds two docker containers, the backend and the frontend: Your first two containers. While browsing the source code of the backend project you find two more binaries that are being build by a Makefile and are used by an admin user. Great, more containers and a new persona that you were previously not aware of.

With the findings at hand, you come up with a container diagram and update the internal wiki - the good boy scout you are. A container diagram shows the individual deployable containers that execute logic or store data. It also records how containers communicate and informs the reader about technical choices.

While looking at the diagram you decide to take a closer into the database to see if each customer has an email associated. You realize that not all of the customers have a valid email address associated and add this finding to the discussion log.

You were hired as a backend engineer and only played around with a single-page application style frontend but being familiar with all the other technologies involved, you feel ok about implementing the feature request. But still, you are glad not to have given an estimate right away.

With the Frontend and the Backend communicating over HTTP/REST with one another, you suspect that the major software change needs to be done in the backend component and you might not need to touch the frontend at all. Thinking about the failure case you realize that the feature request only mentions the happy case. You add a new question to the discussion log about failure scenarios and how the customer should be informed if something goes wrong. Maybe you could use an external email-api provider like sendgrid instead of a traditional email server to get feedback on the delivery status as well.

Component diagram

Confident, that most code changes will happen in the Backend, you start to look at the internal components of the Backend component in more detail. You remember your colleague talking about ‘following a loose clean-architecture’ and are eager to see the structure of the code and building blocks.

Following the entry path of the Frontend making HTTP calls to the Backend, you find the OrderHandler responsible for validating the request, doing authentication and authorization checks, and finally executing the actual task. Tasks related to an Order are grouped in an OrderUsecase. Looking closer at the OrderUsecase you realize that this component has quite a few dependencies all abstracted behind interfaces. Maybe this is what your colleague meant by ’loose clean-architecture’? You do a quick search for clean-architecture, tabs to read later… and start to scribble all the boxes and arrows on a sheet of paper. The usecase makes calls to the AdvertUsecase, uses a few repositories for storing and retrieving data; more arrows.

The usage of interfaces makes you curious and you look at all the implementations of the OrderRepository. Luckily only two implementations are found by your IDE: a in-memory implementation, which is being used for testing, and a mongodb implementation which is used in the production code. This pattern gives you a good sense of how to test the email functionality and you make a note in your discussion log on using an interface to plugin different implementations during testing and in production.

The OrderUsecase is already quite large when it comes to lines of code and you add a note to the discussion log to talk about your uneasy feeling to add another dependency with the other developers. If you would send the email from within this usecase, you wonder if you have all information available:

• Purchase History, Reference Number: The Usecase receives the order details, creates a reference number, and stores it in the repository. All good.
• Email-Address: The OrderHandler passes the username to the call of the usecase - but not the email address. Fetching the whole User Object would mean adding the UserRepository to the list of already plenty depencencies. You definitely have to talk to the other developers.

Looking at the control flow of the OrderUsecase and where to call the ‘SendEmailToCustomer’ function, you realize that you don’t know the content of the email address let alone the subject to use. Must it be an HTML email or will plain text suffice? And how often do those templates change? What is the best place to store those templates? More questions to be added to the discussion log.

With your sheet of paper full of arrows and boxes you create a new wiki page to draw a proper component diagram, start drawing the diagram but soon realize that this might be working in vain. The backend evolves too fast, and this documentation will be out-of-date in no time. Instead, you take a deep breath, pat yourself on your shoulder that you did not give an estimate right away, and grab a coffee.

Discussion with Product Owner

Freshly caffeinated, you turn to your list of discoveries and draft a message to your Product Owner.

You start by letting her know, that you discovered that not every customer has a valid email address and that this field is not mandatory when creating an account. To give her something to work with, you let her know the numbers. You would like to know how the system should behave in the following failure scenarios, … and ask for the template text to be sent as well as the subject for the actual email. You finish your message by offering to go through your findings in detail together.

A few minutes later you are asked to stop working on the feature request. There is uncertainty if the customer is even ok with receiving emails without giving explicit consent. Instead, you are asked to estimate, how long it would take to change the signup process to accommodate the consent and record it. Luckily you already know where to look and how the user details are stored.

You thank the c4 model silently, add the newly created diagrams to the wiki, leave a comment in the ticket with your questions, and set up a call with your colleague to discuss how to reduce the complexity of the OrderUsecase. On your way back home you plan on reading up on the clean architecture as well.

Appendix: The question log

The list of questions and notes taken while doing the c4 deep dive. Not every question ended up being asked or answered.

System Context:

• Should the backoffice persona also be informed about a purchase?
• What are the email server credentials, and how to connect?
• What email address should be used as the sender?

Container:

• Not all customers have a valid email address associated. The system can only send emails to valid email addresses.
• Should the customer be informed when something goes wrong while sending the email?
• Elaborate on the benefits of using an external email-API provider like sendgrid vs a company email server.

Components:

• What is the written email text to be sent? Will it be HTML or plain text?
• What subject line to use?
• How to get the email address from a username? Ask the repository?
• To colleagues: think about breaking down the OrderUsecase into smaller components
• Read later: clean architecture
• How to test: Use interfaces to make tests independent of external dependencies