In an earlier article, I described the idea of a low-value test. In this article I want to elucidate the idea of “value-add” and “value proposition” in the context of testing or testing as a subdiscipline of engineering.

In a previous article I shared the code example below. Click the language tabs to switch — your selection persists across every code block in the article.

That code was not actual production code. But it’s not far off from code found in most code bases. Over the course of this article, we will refine this example into something more meaningful. We will use it to discuss testing, quality, and the idea of “value proposition” and how these are all related.

Purpose of test

What is this test’s purpose? What value does it add? How do we improve the test and its value-add? In order to answer these questions we need to be able to define the heuristic for desirable tests or tests that we place value on. What do we mean by we? “We” should mean You + Team(s) + Customers or collectively the organization’s stakeholders. We does not mean a test that increases a code coverage metric. These stakeholders have different metrics of value.

As a professional, it is your purpose to deliver value to others including the team(s) you work on or for and your customers. So we ask the question from a slightly different lens, what value does this test provide to you and your stakeholders?

How do we define value?

How do we define value here? Well… what does this test do? The test does a few things:

  1. It depends on a component. Maybe this component is needed for a feature depended on by your customer. It’s not clear though. So let’s start doing some refactoring. Let’s change

to

For the purposes of this article, we will imagine we are developing financial software.

  1. Next, we will configure the test to define the behavior of our component under test now named PaymentProcessor. Let’s refactor again.

Now we are getting somewhere. But what does this new code do? Well we first define our setup or “context”. In our hypothetical domain, what is the context at which we can “process an order”? In our domain in order to process an order, we need some data. This includes:

  1. “A payment amount”
  2. We need a customer the payment is related to as a tangential domain actor.
  3. Most importantly we need an invoice a payment is applied to.

Let’s step back. Is Payment Processor the right name for this? We are really updating an invoice.

Let’s continue iterating here.

You may be noticing that we are:

a) designing a domain to allow us to discuss the idea of value, b) working backwards in making a meaningful test, c) maybe a bit of both.

Simply stated, we are now defining a formal domain that reflects a real problem that our stakeholders care about. In traditional test engineering, another really important point is to consider how focus on the quality of the test potentially forces us to improve the actual code. This article won’t focus on the following point: cleaner production code yields more maintainable, less buggy code, and happier customers. This is why we test. This is why understanding value proposition is important.

In the context of business, we call this an opportunity. Our simple code example allows us demonstrate our value proposition or at the very least value to you the reader. They (the customers) care about payments being applied to invoices. They (the humble engineer) care that the payment was processed and also care if they weren’t. Let me clarify, customers do as well. They just assume that their payments are properly processed. It’s on us to provide that value proposition.

Let’s further solidify our example … Our updated test now looks something like:

What are we testing? Why are we testing? What is our approach to testing?

What are we really testing here? Are we testing the controller works? Are we testing payments are processed? Are we actually testing payments are applied towards an invoice? If you said no 3 times, you are right. If you didn’t, let me explain.

What is the purpose of a controller? My arbitrary explanation is that controllers control the flow of data from the customer to our app. In reality, controllers are a boundary layer between some form of API communication (e.g. REST) to our app (Java code ). My initial response wasn’t that far off. The controller is a boundary.

The framework is responsible for the wiring of all of that. And all of that is a lot:

We could make the controller also responsible for invoice business logic. But that’s not considered good design at least according to SOLID. Each class or component should be responsible for only one thing. Right now that thing is controlling the flow of data from our external boundary to our corresponding request handler function defined on the class. We probably want to make sure that works. But what is “that”? That is a lot. In practice there are a few different ways of doing this. We could do what we did in our above example: instantiate the controller and call the request function. But what are the side-effects of this design? Well a few:

  1. We now coupled our test to the exact controller responsible for invoice payment processing logic
  2. We are not validating that our controller is properly wired. The test can pass and it does if we wrote it “right”. But a passing test does not really validate that its purpose (or contract) is actually adhered to. What does this mean to our value proposition?

At this point it may feel we have more questions than answers. That’s okay. Let’s start with the “what”? What are we trying to exactly do? What is our contract with our customers?

What is a contract?

A contract is an agreement that specifies certain legally enforceable rights and obligations pertaining to two or more parties. A contract typically involves consent[1] to transfer of goods, services, money, or promise to transfer any of those at a future date.

From wikipedia. How does this apply to software? Actually a lot. Software is an agreement between a user and author(s). Software is about providing a service to customers, users consenting to that exchange and potentially exchanging money (or other goods (e.g. data)). In our above example, the contract is roughly about processing orders, handling payments, receiving invoices or a combination of the three.

What does this have to do with testing?

Now we are asking the important questions. Inherent to providing any service is 1) its value-proposition. It is what convinces a customer to actually engage in the contract and 2) the quality deemed from that exchange. As a business there are ways to guarantee that quality to maintain (or hopefully advance) that value-proposition. A few ways to do this 1) A QA Department 2) Clear Specifications 3) Tests 4) A cohesive platform that reinforces each other towards a strategic vision of a perceived value-proposition that a company can sustain a competitive advantage on. The last one is hardest to achieve in practice.

Now before I derail too much, let us focus at the micro or nano-level here. What really matters at the end of the day is does it work. In engaging in that contract, the internal implementation doesn’t matter to the customer. It does to you as an author. Software may be a craft for you. Cleanly written, well-designed, understandable system(s). Maybe your craft is empathizing with customers and understanding their needs and translating those into technical requirements. Maybe your craft is building a platform to enable others. That is your value proposition. Take pride in your value proposition. Tests are a way of guaranteeing that value-proposition. Tests come in many different shapes and sizes. It could be manually testing right after a deploy and doing a quick sanity check. It could be micro-benchmarking an algorithm that powers the payment calculation logic. It could be getting customer feedback on proposed features. It could be automated tests that validate your APIs work. All of these actions reinforce your and your organization’s value proposition. By reinforcing your value proposition, from a business perspective you either maintain or enhance your competitive advantage.

We want to put bread on the table and take care of our families. We also want to make sure that we can sustain that security.

Tests help with that. I’ve discussed a few concepts here in the abstract. I’ll bring up one more that I will go into more detail in another article. Tests help establish feedback loops. Feedback loops provide us an opportunity to learn from our mistakes, incorporate feedback, and be better. Tests and their various forms are just a more formal way of doing this. In the abstract tests are about validating the contract we promise others.

At this point, outside of a formal definition, I haven’t even defined “what” our value proposition actually is. Let me do that right now: It depends. This level of professional discretion is important. Are we providing an interactive front end app? Are we ingesting data in a ML pipeline? Do we have a simple CRUD app? All of these will have vastly different requirements, purposes, and contract(s). In these scenarios, your stakeholders will have different needs. As a professional it’s on you to apply judgement on understanding those needs and how to provide value in these exchanges.

This professional judgement is important for several reasons. But first we must acknowledge that professional judgement is gained over time with experience. It’s not sold or taught in skills. You may gain some with a strong professional network but you gain it with experiences “in the trenches” on the job. This professional judgement provides you the ability to discern value-add and activities that detract.

So professional judgement may provide insight to know that:

code like this means we haven’t actually validated our contract, our promise to our customer. We don’t know that InvoiceService#applyPaymentTowards actually works when you configured your test to return some predefined value.

Some of you may be reading this (if you still are thank you) and disagreeing.

  • “But I confirmed that there is a function called applyPaymentTowards”
  • “Aaaannnnd I confirmed that it works within the controller”.
  • “And it’s always been done this way”

All respectable opinions. But our requirements just changed. The logic on what happens when a payment is processed changed. You rush to fix it to be compliant with new legislation meant to help small businesses that depend on our software. You changed the function to accept the new arg (arbitrary example):

Your tests pass. Your manual testing works. Your junior QA misses it. Your AI Agent of choice gives you the 🟢 light to move forward. Your senior QA is out fighting fires on another product. You deploy to production. You drive home and enjoy some dinner and spend time with family. You get a slack notification a few weeks later. Invoices made in the last 2 weeks are all wrong. You start getting multiple slack notifications. All of this after putting out unrelated fires on another part of your domain. You pause. You contemplate life decisions. Maybe the trade would’ve been better.

What-If

What if instead of focusing on passing tests. Or getting the build out. We focused on making sure the tests actually expressed the needs of the customer. What if the test actually validated your changes. What if the tests actually validated the implementation.

That scenario above? That’s not hypothetical. That is what happens when the test’s value proposition is not well defined but they are passing. The build passed. The contract didn’t.

In a follow-up, I will cover examples that show how we extend the above test. Make it more complex. And more generally look at approaching your tests to focus on what matters most to your stakeholders.