Monday, January 19, 2015
Let's talk about DevOps!
Imagine you're a sysop. Your systems are up and running, so you sit back and chill.
Yeah, I said, "Imagine". Now, I mean, seriously. Not like you'd have time for that. Every platform requires frequent maintenance, you wouldn't want any service to deteriorate. While you're juggling security updates, intrusion alerts, overflowing storage, network load and whatnotever could possibly go wrong on the average working day, those nasty Scrum teams come around and ask for a configuration change, giving you only a couple days to implement.
They can't even formulate a proper Request for Change including a detailed risk assessment and simply want you to "install that one module" and because you think that the new storage module is more important, they request "just give us the root password and we'll do it ourselves".
Anyways. That's beyond reasonable. Someone will compromise the server in one way or another and everyone will point fingers at you. Worst case, you'll get the sack. No chance in hell freezing over!
Ok, that was "Why are admins always so slow" for all of you developers out there.
Now, let's imagine you're the developer. You've got your story backlog, you've got the code lined up and are ready to deploy. Inside your Scrum team, you can simply prioritize the issue, and everyone will swarm to solve it within the business day. Unfortunately, this time you need a change to the system configuration, a certain module needs to be installed. And that means you have to go into the lion's den of sysadmin. And probably they'll ask to have everything written up formally, only to shove it into some tracking tool where it will rot until a date far beyond sprint review. Chances are, your story won't make it, because they'll be too slow.
That is the flip side of the coin. Now, who is wrong?
This dilemma exists in many companies and the result is neverending fingerpointing and blame-games. But - that's not how our story needs to end.
How can you synchronize Development and Operations without infringing on either, while satisfying both? What can Development offer to Operations to make their life easier, how can the mutual pain be eased?
In small startups, you can simply move the admin role inside the sphere of responsibility for the Scrum team, but as soon as the organization grows, difficult questions will arise: How do we keep servers secure across teams, how do we maintain dozens of individual systems which can go berserk every minute, etc?
Some people go as far as claiming "#NoOps: Developers can do everything!", but let's be honest. You're not very likely to find someone who can do the highly specialized fulltime job of a developer and still be a highly specialized fulltime admin, while at the same time having time to keep up with new business requirements and user activity on the platform - and still working on an affordable payscale. On the other hand, distributing administration within the team may simply not be feasible, because too many half-competent (no offense) developers run a risk of making a dangerous modification towards the servers.
The main result I have seen so far coming out of #NoOps claims is servers that are hardly maintained, with configurations that are insecure, incomprehensible and run a tremendous risk of falling apart any minute. Many developers don't even understand why they shouldn't simply log in as root. And that's just the tip of the iceberg. It goes further over using telnet login, creating tablespaces with "autoextend" and more. (I apologize to all the admins out there for the head bruises incurred by facepalming too hard and to all the developers out there who don't understand where the problem is).
Development must understand the pain of Operations rather than pointing fingers - and Operations must understand the pain of Development.
DevOps are the solution: Let's just admit that administration is necessary and that developers are developers. And then work out solutions from there. Rather than usurping admin activity into the dev team, let's facilitate a positive information exchange, foster mutual understanding and simplify the interaction.
For instance, you can automate the configuration management with CI, or you can go as far fully automate deployment and eliminate the risks that always come with major releases. Development can relieve Operations of any activity that really no admin wants to do (i.e. working through tedious, poorly written instruction manuals) , and in turn, Operations can provide platforms where change is smooth to implement and easy to track.