Let's take a deeper look at CI:
CI is much more than just a tool! |
Process
Let's ignore for a second the seemingly trivial "After a merge, automatically build and run tests, then report failures" aspects of the CI process and dig into the real question "What do we actually expect from our CI process?" - what do you expect from your CI process? That should be, first and foremost, "instant feedback". But: Feedback on what? Based on what feedback you are looking for, you must design your own CI process to give you the most important feedback clearly, quickly and reliably. So, the difficult question becomes "How do we get that?"Scope
In order to answer the question, "What feedback are we looking for?", we first need to answer the question "What does Integration actually mean?"Are we trying to integrate:
- pieces of code into a repository?
- features into a system?
- activities into processes?
- processes into process chains?
- components into a larger system?
- systems into a larger ecosystem?
Just as an example, let us look at the question "Is our User Account functionality OK?" in comparison to the question "When 10000 users are on our site at the same time, will everyone be served without encountering problems?" - you obviously need a completely different approach to get feedback here.
Before you can answer how you want to get the result, you must define what result you expect and why you need that information.
Test Suite
Once you know what you want and how you want to get it, that obviously just needs to get done. However, it shouldn't just "get done", but it should get done well. Common problems in CI test suites are:- Poor coverage, i.e. the information "it passed" is not meaningful
- Flakiness, i.e. the information "it failed" is not meaningful
- Interdependence, i.e. the information "27 tests failed" is not meaningful
- Slow tests, i.e. the information "It finished" is no longer meaningful
Frequently asking "Does our test suite solve the problem we have?" is quite important. While adding tests as needed is easy, both weeding out unnecessary tests and refactoring for optimal information value are often tough calls.
Mindset
The biggest problem with CI in many organizations is that they do not realize that CI is neither a tool nor a process - but a way of working supported by these. Developers do not serve the CI tool and their job is not to keep the master green. As long as developers consider CI a burden rather than a help, then no specific implementation will yield the desirable benefit. Much rather, CI helps developers get their job done well and a red master is an indicator that something is wrong.Good CI will not only fix a Red Master, but trigger Continuous Improvement [also "CI"] when the Master goes Red: "Why did it turn Red? What can we do to avoid that situation in the future?"
Working on a proper CI mindset goes a long way in successfully implementing CI. A key aspect of this mindset is frequently asking "Which problem are we actually trying to solve with CI? Do we do that?"
Tool
A fool with a tool is still a fool: The tool must be adequate to solve the problem at hand and set up properly to deliver the intended benefit.Initially, CI tools are empty shells that trigger build jobs and report Pass or Fail. Configuring a source code repo and defining the test suite to execute is done quickly. But you may want to set up individual build chains for different problems, depending on what you are trying to achive. What these chains do, how they interact and what you do with the resulting information is something the tool can not help you with.
No comments:
Post a Comment