Friday, March 17, 2017

Building a CI tool from scratch in one hour

Is a CI server the right answer for your team? Do you need a CI tool? "Well, that's obvious!". Is it? You may be surprised! 
This is a lesson I learned about the Agile Principle #10: "Simplicity - maximizing the amount of work not done - is essential." 

A simple CI chain, providing essential feedback to the developer

We need a CI tool

In most software development projects, someone will say the sentence "We need a CI server" - usually right at the start of development, before the first line of code is written. In an extreme case, a company paid many thousand Euros to set up a CI chain before any developer was using it. Tools like Jenkins, Hudson, Bamboo and the likes pop into mind right away. They are very good at what they do.

When I worked with one organization that had been using Atlassian's Bamboo from day 1, I talked to a senior developer who took personal responsibility to create the best possible CI process that developers could get. He jested that setting up a build chain in Bamboo required a master's study in Atlassian tooling.

This leads me to pose the question: "Do we need the complexity that a CI server creates to achieve the goal we want to achieve?"

CI isn't rocket science

In its simplest form, CI doesn't do more than retrieve the latest commit from the version control system, create a new build, deploy it - and run the tests. Developers are informed about the consequences of their actions, and that's it. Of course, complexity can be added as needed.

Do you really need a magnificient tool to do this? Is it worth sinking hours (or days) into configuring and maintaining a platform to do this trivial job for you?

My claim: You don't need a standard out-of-the-box CI tool!

How to create a CI tool

We were in a training setting. It was a Certified Scrum Developer course provided by Andy Schliep. It was fun and we were building a training application (real, working software!) in Ruby. Andy set a constraint for the Definition of Done "Works on this machine".
There was eight of us in the course creating Ruby code - so we needed Continuous Integration. In a training setting, we didn't have a CI server, nor the time to set one up. So, we reduced it to the essentials. "What do we really need from CI?"
A developer story was born.
Since we had a cool gadget from blink1, Andy added the Acceptance Criteria here: "The light must be red on a failed build and green on a successful build." Blink1's API is very simple, so we got going.
Even if we had used Jenkins, the integration with our blink1 USB light would still be an open point.

Since I love Shell, I started writing some Shell code to access the blink1 LED:
  ~/tools/blink1-tool "--$1" 2>&1 >/dev/null
This small function snipped would allow me to set the color of our blink1 LED to anything I wanted, for example, red, blue - or green.

The next thing I needed was the failure state:
  blink red
  echo "CI step failure." | log
  exit 1
Of course, I'd also need a success state, which looked pretty much the same.
Like that, I could already "fail the build".

We then brainstormed which activities we would need in our CI. We ended up with the following sequence of actions:

  1. checkout the latest code from github
  2. install the bundle
  3. clean the database
  4. run the tests
  5. start the server

If all five activities succeeded, we would call this a successful build and "pass the build".

The main control logic of our CI looked like this:

  for task in checkout install_bundle clean_db run_tests; do
  do_task $task
  ( succeed )
  do_task start_server

Of course, we still needed to fill the logic for each task step, so we did that, too. Tapping into the knowledge of my teammates as to what exactly needed to be done, I wrote functions for each task as well. It looked pretty much like this:
  git_feedback="`git pull`"
  echo "$git_feedback" | log
  [[ "$git_feedback" == "Already up-to-date." ]] && return 1
  return 0

After we had the essentials down, we started polishing the code a little, adding extra features such as turning the light blue while the build was in progress.

Everything together, we spent roughly one hour to create this "CI tool" which served it purpose. It is less than 100 lines of code and can be viewed here. Of course, I should have added unit tests - but we were content with doing every unit test manually, because this was only a training exercise and I didn't have my Shell unit testing framework at hand.


You don't need to add the complexity of "yet another platform to maintain" to your development unless there are pressing reasons. Try looking into this question: "Who needs what - and what is the minimum we need?" Talk with the team, discover the real need and find the simplest way to reach your purpose.

Only add extra tools when you have sufficient evidence to prove that a few lines of code can't meet the same purpose. Remember: Every tool adds complexity and work.

No comments:

Post a Comment