Low-Code, No-Code, Full-Code - The Testing Challenge

In the Enterprise world, there's a huge market for so-called "Low Code" and "No Code" Solutions, and they do have a certain appeal - you need to do less coding, and as such, need less developers, to achieve your business objectives, because they bring a lot of "out-of-the-box" functionality.  So why is it even something to talk about - and how does that relate to "Agile" ways of working?

Let's explore this one from a quality perspective.

The Paradigms

No-Code: Configure and Go

"No Code" solutions are especially appealing to organizations that have no IT department and are looking for something that someone without IT knowledge can configure in a way that's immediately useful.

An early implementation of no-code platforms will typically not even include a staging environment where people try out things. Many times, changes are immediately live, on a productive system. That's great for small organizations who know exactly what they're doing because it absolutely minimizes effort and maximizes speed.

It turns into a nightmare when someone, somehow, by pure accident, managed to delete the "Order" object and now you're happy-hunting for a couple thousand unprocessed orders that your angry customers are complaining about - with no way to remedy the system.

And it turns into an even worse nightmare when the system doesn't do what it's supposed to do, and you've got a chance smaller than hell freezing over of figuring out why the black box does what it actually does instead of what it's supposed to do.

When introducing Quality Assurance on a No-Code platform, organizations are often stuck using third-party testing software that uses slow, flaky, difficult-to-maintain, expensive UI-based tests which will eventually get in the way of high speed adaptability. Clean Code practices applied to testing are usually a rare find in such an environment.

Low-Code: Configure and Customize

"Low Code" solutions are especially appealing to managers who are out to deliver standardized software to their organization fast. Many of these systems bring huge chunks of standard capability out-of-the box and "only need customization where your organization doesn't do what everyone else does."

That sounds appealing and is a common route in many organization, who often find out only years after the initial introduction that "you can't sustain your market position by doing what everyone else does" - your business does require a lot of customization to stand out in the market, and the platform often doesn't accommodate for that. 

Most vendor solutions don't provide a suite of functional tests for your organization to validate the standard behaviour, which means you often end up creating duplicate or highly similar code in your customization efforts - or use standard functions that don't do what you think they would. Worse yet, many use proprietary languages that make it very difficult to test close to the code. In combination, that makes it extremely hard to test the customization you're building, and even harder to sustainably keep the platform flexible.

Full-Code: Design, Build, Optimize

"Full Code" solutions sound like the most effort and the slowest way of achieving things. But looks can be deceptive, especially to a non-expert, because a modern stack of standard frameworks like Spring, Vue and Bootstrap, can literally make it a matter of minutes for a developer to produce the same kind of results that a low-code or no-code platform configuration would provide, without any of the quality drawbacks of Low-Code or No-Code.

Your organization has full control over the quality and sustainability of a full-code solution. It depends entirely upon what kind of engineering practices you apply, which technologies you use and which standards for quality you set for yourself.

Quality Control

To sustainably high quality at a rapid pace, you need full quality control:

• You must be able to quickly validate that a component does what it's supposed to do.
• You must be able to quickly figure out when something breaks, what it was, why and how.
• When something breaks, you must be able to control blast radius.
• You need a systematic way of isolating causes, effects and impact.

The most common approach to maintain these is a CI/CD pipeline that runs a robust test automation in the delivery process. To make it feasible that this control is exercised upon every single change that anyone makes at any point in time, it should not take longer than a few minutes, lest people are tempted to skip it when in a hurry.

The problem with both No-Code and Low-Code solutions is: In many cases, such platforms aren't even built for testability, and that becomes a nightmare for agile development. Instead of running a test where and how it is most efficient to run, you invest a lot of brainpower and time into figuring out how to run the test in a way that fits your technology: You have subjected quality to the technology, instead of the other way around!

In a low-code environment, this can become even more problematic, when custom components start to interfere with standard components in a way that is unknown and uncontrollable in a huge black box.

Non-functional Quality

Although I would not outright suggest to opt for a full-code solution (which potentially is not in the best interests of your organization, and it's entirely implausible without skilled developers), I would like to share a list of non-functional quality attributes that may not be considered when selecting a new system, platform or service.

In order to remain agile - that is, to be able to quickly, effectively and easily implement changes in a sustainable manner - your platform should also accommodate for the following non-functional quality requirements:

Factor	Decisions
Testability	How much effort is it to test your business logic? This must go far beyond having a human check briefly whether something works as intended. It needs to include ongoing execution, maintenance and control of any important tests whenever any change is made. And remember: any function you can't test may cause problems - even when you're not intentionally using it!
Traceability	How closely are cause and effect related? You don't want a change to X also to affect Y and Z if that wasn't your intent! Are you able to isolate changes you're making - and are you able to isolate the impact of these changes? This should go for the initial setup as well as for the entire lifecylce of the product.
Extensibility	How much effort does it take to add, change or remove business logic? Adding a form field to a user interface is a start, not the end. Most of your data has a business purpose, and it may need to be sent to business partners, reported in finance, analyzed in marketing etc. How much effort does it take to verify everything turns out as intended?
Flexibility	How often will you be making changes? If you're expecting a change a year, you can permit higher test efforts per change, but when you're looking at new stuff in a weekly manner, you could be overwhelmed by high test or change efforts, and cutting corners will become almost inevitable.
Security	Can you really trust your system? Although every system could have vulnerabilities, and standard softwares tend to have fewer, but how can you test for Zero-Days unless you can fully test the intricate inner workings? Also, some legislation like GDPR forces you to expose certain data processings, and you may need to provide evidence what your system does in order to do that. This is extremely difficult when some behavioural description of certain aspects are a black box.
Mutability	How much effort would it take to migrate to a new platform and decommission the current platform? When you introduce a system without having an understanding of how much time, effort and risk is involved in a migration or decommissioning initiative, it might be easier to kill your current company and start another business than to get rid of the current technology. That means you could find yourself in a hostage situation when the day comes that your platform is no longer the best choice for your business, and you have no choice except continuously throwing good money after the bad.

As a general rule of thumb, low-code and no-code platforms tend not to emphasize these, so the value your organization places on these non-functional requirements correlates with the plausibility of selecting this approach.

Conclusion

With a lot of these to be said, if you're in the comfortable situation of introducing a new technology, ensure that you check the non-functional requirements and don't get blinded by the cute bucket of functionality a low-code or no-code solution may offer. If your platform does poorly especially on traceability, testability or mutability, you're going to trade off your agility for some extremely painful workaround that could increase the Cost of Ownership of your solution beyond feasible limits.

It wouldn't be the first time that I'd advise a client to "trash everything and start with a blank folder. Within a few years, you'll be faster, have saved money and made better business."