In part 1 of this exciting 2-part blog series, I argued, quite elegantly I think, that DevOps certification is perhaps not the single greatest breakthrough in the advancement of software delivery since the invention of computing. In this part, I’ll attempt to go into even further detail in support of my hypothesis…
At an agile conference last year I did a quick survey to see what people valued the most about Agile Cerification. The results were conclusive, not one single person said they valued the certification itself. Most people said they thought the training was the most valuable thing, and I can understand that 100%. Quite rightly, people valued the knowledge they had gained and the content they’d been taught over a certificate they walked away with.
But certification is still very popular, and that’s hardly surprising when you see how many people advertise roles for “certified” scrum masters. But the abundance of certified scrum masters doesn’t seem to have done much, in my view, to help progress the agile movement as a whole. In fact, I’m more inclined to believe that some “agile” certification has done more to confuse and derail the agile movement than to actually help it.
Organisations think they’re agile because they’ve hired some certified scrum masters (or sent some people to get scrum master certification). I’m sorry, but hiring a certified scrum master makes you no more agile than hiring a violinist makes you an orchestra.
I’m not going to try to make excuses for misunderstanding the very meaning of “agile”, but it’s pretty easy to see how some people might think “well I’ve now got these Scrum Masters for christs sakes, MASTERS – not just any old scrum practitioners, and they’re certified! So if that doesn’t make us agile then why did I spend so much money on sending them on that certification course!?!!” (Answer: because you stoopid). Scrum certification is “reassuringly expensive” (genuine quote right there), and is VERY useful for teaching people about how to run scrum, but it doesn’t make you agile. Don’t be fooled by the price tag and the highly egotistic “Master” title.
As part of my role as a consultant, I sometimes get asked to assess organisations agility. More often than not I get told “Oh we’re agile, we do scrum”, or “Oh we’re agile, we do sprints and have stand ups” and that sort of thing. These things, sadly, don’t necessarily make you agile. They’re just things. But one big problem in the software delivery world right now is that many people DO think that those things “make you agile”, and that’s all there is to it.
The silver lining for me of course, is that I get quite a bit of work out of it! I get to help some of these organisations realise that thier ability to do scrum, and their overall agility are two quite separate things.
And this brings me round to DevOps.
The DevOps world has an identity crisis every bit as bad as the agile world. Where agile suffers from people confusing the link between “agile” and scrum, in DevOps we suffer from people believing DevOps means Automation.
To be clear, DevOps doesn’t mean Automation. There’s already a word for Automation, and that word is Automation.
For me, DevOps is about the way in which teams work in order to create high quality products from Operational and Development perspectives.
Sure, automation can play a part in that, but it’s only a part.
Part of my role is to help teach people about how to unlock the power of DevOps, and to do that I usually have to start by explaining what DevOps is and what it isn’t. We call this the “Education” piece, because that’s exactly what it is – it’s not simply a case of defining something and drawing up a glossary, we’re literally opening people’s eyes and minds to what DevOps actually is, and what it can do for organisations who do it properly.
At one point we even thought about formalising this DevOps Education and even providing some sort of certification or professional credits system, but we quickly realised that this was utter bollocks.
The thing is, you can’t certify something that doesn’t have a commonly agreed definition, and you can’t certify a philosophy. You can certify actions and behaviours, and how well people understand particular frameworks (like scrum for instance), but you CAN’T CERTIFY A PHILOSOPHY. And anyway, who are these self-proclaimed guardians of the DevOps philosophy? These people who are so sure that they not only understand DevOps better than the rest of us, but also believe they’ve stumbled upon the perfect training program for passing this most precious wisdom on to people who attend their course? It must be a veritable who’s who of the DevOps movement, the Adrian Cockrofts, Jez Humbles, Patrick Dubois and John Allspaws of this world… Hint: it isn’t.
It’s almost as if the whole thing is just a scheme to make money! Imagine that!
So to conclude, I fear that DevOps certification isn’t worth the paper it’s written on, and bandying around a DevOps certification is only going to perpetuate the problem that we already see within agile, where organisations will believe that they’re “doing DevOps” simply because they’ve hired people with some bullshit certification.
DevOps Certification is complete bollocks.
I’ve recently been working with a number of scrum teams, across a few different organisations, and I’ve started to notice a bit of a trend with regards to agile practices dropping by the wayside. Now, this might be a sweeping generalisation, but I’m noticing something of a correlation between Scrum teams who are struggling, and the “disappearance” of Sprint Goals and Backlog Grooming. Even with ScrumMasters around, these 2 practices seem to be the first to bite the bullet, which makes me wonder if their importance is not as well understood as it could be…
If Scrum “good practices” were an episode of Star Trek, I imagine Stand-Ups would be Captain James Tiberius Kirk, Retrospectives would be Spock, Sprint Planning would be Scotty, and so on until we’ve covered all of the main characters & characteristics. Now let’s imagine the crew of the enterprise are assembling a landing party, to investigate a strange new world, home to as-yet unknown and possibly hostile alien life forms. The usual suspects are on the holodeck waiting to be beamed down, along with one new lower-ranking character wearing a red shirt – yes, say hello to the Sprint Goal.
If you’re not familiar with the whole “red shirt” thing with Star Trek, it goes a bit like this – in the original Star Trek series, whenever the crew beamed down to a hostile planet, they were seemingly always accompanied by a lower-ranking disposable crewman wearing a red shirt, who would promptly die in some sort of fight with an alien life form. It was always the red shirted guy. If you’re interested and fancy a bit of a giggle, this website actually contains a statistical breakdown of how likely a “redshirt” is to die. Brilliant stuff.
Anyway, back to the Sprint Goal – it appears that rather like the redshirt, when the going gets tough, the Sprint Goal appears to be the first one to cop it. This isn’t wholly surprising, because the Sprint Goal doesn’t scream “I’M IMPORTANT AND DELIVER VALUE” in quite the same way as stand ups and retrospectives do. But they are important and they do deliver value. Sprint Goals are rather like acceptance Criteria, in that they ensure that we are doing the right thing. Bear with me on this…
- Any project or product will have an objective
- The objectives get transformed into a plan
- The plan gets split up into milestones and iterations
- The milestones and iterations are made up of sprints
- The sprints contain numerous stories
Hopefully this little list will help to demonstrate how detatched a story can be from the original objective. Sprint Goals are a way of making sure our stories are all aligned with our project or product’s objectives, and not just a collection of seemingly misaligned tasks.
If we just start plugging away at stories without having a goal or objective in mind, then we’re not really giving ourselves the right amount of context to make the best decisions.
I sometimes see this issue more clearly on so-called “self-organising” teams, where everyone seems to be sprinting, but not necessarily in the same direction. Deciding on a Sprint Goal before your spint starts is a great way to ensure that everyone is collectively sprinting in the right direction.
Sometimes people tell me that it’s really hard to define one single sprint goal for their sprint, and so they don’t bother with one at all. In this instance I would recommend setting more than one sprint goal! Working towards 2 or even 3 complimentary targets is surely better than not working towards one at all.
Backlog grooming is the other practice that I see slipping. I’ve even seen a few teams who just stop doing it altogether. By backlog grooming I mean making sure the backlog is relevant and up-to-date, and that the upcoming stories are of an acceptable standard.
When backlog grooming starts to deteriorate I see scrum teams really struggling. Firstly they struggle to get through planning because the stories are not broken down or thought-through sufficiently. Secondly they struggle to work on the stories because they aren’t of an acceptable standard, thirdly they get interrupted with re-work from the previous sprint (because the stories were unclear and so they built the wrong thing), and fourthly the backlog becomes a big scary mess of out-of-date stories and only the Product Owner knows if they’re still relevant or not.
As you can probably see, failure to properly do backlog grooming can seriously impact a team’s ability to deliver high quality solutions on time. This is why backlog grooming is so important.
So what can you do if you see backlog grooming slipping? It’s easy to say “Just make sure it gets done”, but that just doesn’t work. I’ve also seen senior managers forcing stories into a sprint knowing full well that the stories weren’t of an acceptable standard. The result is the same, stories that are unfit for consumption.
If this situation sounds at all familiar, I recommend adopting a more disciplined approach to accepting stories into a sprint – this is your first layer of protection against poor-quality stories. I encourage the use of the INVEST principle (or rather the NEST principle – I’m not too bothered about the I, and the V).
If you’re not already familiar, here’s what INVEST stands for:
- I is for Independent, meaning each story should not have to rely heavily on any others. I ignore this one simply for a happier life
- N is for Negotiable, but I like to think it just stands for Now Go And Talk To Someone, because I like to use it to remind people that a story is the starting point for a conversation, not a massive requirements document!
- V is for Valuable. This one’s straightforward, but for me a bit superfluous. If a story isn’t valuable then what’s it doing on the backlog on the first place?
- E is for Estimable. If the story is impossible to estimate to any degree of confidence then it’s too risky and needs breaking down or time-boxing.
- S means Small. Sort of related to Estimable. But if a story is too large then that should set off alarm bells. Smaller is better when it comes to stories.
- T stands for Testable. Yup, I’m afraid we’re going to have to think about how we’re going to test the story!!
You could try to make sure that your backlog is split up into 2 pots of stories, those that have been INVESTed (or NESTed, if you’re more like me) and those that haven’t. Be clear that you can only accept stories from the NESTed pot. If you find yourself in this situation and you’ve adopted the NESTed approach, then let me know how it goes!
Working remotely is so common these days, that I’d say the vast majority of organisations I work with today accommodate some degree of remote working. I think it’s great that organisations are prepared to do this for the sake of their employees – it shows an awareness of that so-called “work-life balance” that so many of us have managed to get wrong in the past.
It’s perhaps not surprising then, that when I speak to people about the importance of culture and collaboration as essential ingredients in any devops or agile transformation, people become concerned about how compatible this is with their working-from-home policy and globally distributed teams.
Unfortunately I’d say there’s no straight answer. As with most things in DevOps, it depends on many factors. We all love a good list, so here’s my list of things that can impact your DevOps journey if you’re working with distributed teams and remote workers:
- Team maturity (is there strong trust within the team?)
- Decision making (how effective are you at decision making?)
- Location (do your working days overlap much?)
- Language (do you all speak and understand the same language effectively?)
- Collaboration (It’s not the same as communication)
- Management techniques (are you a command and control freak?)
- Tooling (are you on mute?)
Bearing these factors in mind, let’s take a look at what makes an effective distributed team (in my humble opinion, of course).
High trust between individuals in the team means people are comfortable allowing others to work autonomously, safe in the knowledge that they will deliver what they committed to. High trust also means you’ll feel comfortable asking for help when you need it. In high-trust teams, a daily stand-up is usually sufficiant for a Team Lead (Scrum Master, PM, PO, or whoever) to feel comfortable and confident that the individuals can be left to get their work done. This is not to say that they will work alone, far from it, to do DevOps successfully you absolutely MUST collaborate and work together effectively throughout the day, but crucially they don’t need a Team Lead to continually check in on them to make sure they’re doing the right things.
Devolved Decision Making
Responsibility, accountability, empowerment – all of these are high-scoring bullshit bingo words, but they’re also important factors in effective decision making. Give team members the power to make decisions without having to summon a committee meeting and things will run much more smoothly. If this idea scares you, then mitigate the perceived risk by ensuring all decisions are retrospectively reviewed – doing this allows people to go ahead and get on with their work but it also allows you to catch any bad decisions before it’s too late. When it comes to decision making within the team, my policy is to seek forgiveness, not permission. Of course, there’s a boundary within which we all need to work when it comes to decision making, and decisions around features etc should always be made by the Product Owner, but we all knew that, right? Obviously I’m not suggesting that individuals should unilaterally decide to change the language the product is coded in, or introduce a new feature – common sense still has a large role to play!
Most of the successful distributed teams I’ve been involved with have had a significant amount of overlap in terms of the hours worked by the individuals. The least successful teams have had very little overlap in working hours. If you work in the UK and you have significant parts of your team in places such as the US West Coast, Australia, China or Japan then you’ve probably already felt the frustration of having to wait an entire working day for an answer or response from a colleague, only to find that it wasn’t what you needed. In the fast-paced IT world of today, many of us can ill afford that sort of delay, so teams have worked out new ways of dealing with the challenge, such as working different shifts, dialling in to meetings at unsociable hours and so on. It’s often not an ideal solution but if it helps the team work more effectively while allowing you to continue to enjoy a comfortable and convenient working lifestyle then it’s probably worth the effort.
In a DevOps environment you’ll want to make sure that there’s as much overlap as possible between your developers and infrastructure engineers – these are the roles that need to collaborate most closely, so an environment where ALL your devs are in one location and ALL of your infrastructure team are the other side of the world is going to be a real challenge.
Ok, I’ll be blunt – you all need to speak the same language, and you need to do it well. We all know it’s hard work trying to communicate with people who don’t fluently speak the same language as you, and in the end you just end up making excuses for not communicating with them (and that’s a bad thing). It’s high-time we all agreed to speak 1 global business language, and that language should be Welsh (because it’s by far the most awesome language in the world).
For me collaboration is about people working together in an effort to build something mutually beneficial. It’s not the same as communication. Collaboration means you need to be able to listen to other people, make appropriate changes, help others, coach people and share ideas. Tools like GitHub (along with the Git workflows) are great for allowing us to collaborate when working on code. Teams with good collaboration techniques and processes (code reviews, retrospectives, workshops etc) tend to become higher trust teams as well. I haven’t stopped to think why this happens, but it’s an observation. These teams handle distributed working easily because they have such a high degree of interaction anyway, that location becomes insignificant. In a successful DevOps environment both developers and infrastructure engineers will collaborate and use techniques such as pairing and code reviews to learn from each other and improve.
Again we’re looking at high-trust teams. Teams where management are happy to give the individuals the space and time to work are more effective than teams with managers who feel the need to constantly check in on them. In my experience the best style of management to work with a distributed team is one of high-trust and devolved responsibility – one where management provide guidance and support rather than instructions. If you see yourself as a command-and-control style manager or obsessed with micro-managing individuals then you’re probably going to struggle working with a distributed team.
There’s loads of tooling out there to help people work remotely. Most people are already using things like Slack, HipChat and Skype because they are such effective communication tools – but communication is only part of the picture. As I mentioned earlier, GitHub is a great collaboration tool for anyone involved in coding (so devs and ops alike), but we also often need to share large binaries (such as PDFs, Presentations, diagrams, pictures and so on) which don’t usually belong in source control alongside your code. For these types of artifacts tooling like Google Drive and Dropbox are great (as long as your corporate security policy will allow you to use them). I like the latest Atlassian tools for managing requirements and handling wikis because the real-time updates work really well with people working remotely, but in terms of sheer simplicity and ease of use, you can’t look any further than Trello for task management! I’ve seen IdeaBoardz being used very effectively for brainstorming and sharing ideas across a distributed team – like Trello it’s a really easy-to-use and fun collaboration tool.
So, in summary, doing DevOps in a distributed team can be an absolute doddle or it can leave you dead in the water – it all depends on how mature your team is, what sort of management you have, the tooling available to you, the communication skills of the individuals, and your team culture.
In order for any sort of process, framework or methodology to succeed in the IT world, it absolutely must involve a large number of acronyms. And devops is no different. In the devops world we like to say that there are five underlying principles of devops, and they’re represented by the acronym CALMS. As with any good IT acronym, you start with the acronym itself and then work backwards from there. The main reason why the CALMS word was chosen for the devops world was because of the unlimited marketing oportunities it offers. For instance, you could use the “Keep CALMS and carry on” slogan and plaster it all over anything that you can actually print onto, like T-Shirts, mugs, foreheads, powerpoint presentations etc and so forth.
The next trick was to work out what the CALMS should stand for. This was the hard part, and required the input of some of the smartest minds in the devops world to come together and use their collective brainpower to think of some devops words that would conveniently fit the CALMS acronym. So, in May 2013 or something (lets say), some people with names like Gene, John, Jeremy, David, John and Adrian all got together at the top of a mountain in North Wales and meditated on the CALMS acronym. That didn’t work, so they all got really drunk and that’s when they came up with what we now know as the five pillars of devops:
C and S stand for Cats on Skateboards
As everyone knows, the vast majority of the internet is made up of billions of pictures or videos of skateboarding cats. That’s why the internet is so big (and that’s also where the term “BigData” comes from). Devops is all about deploying pictures of skateboarding cats to the internet, in order to satisfy the world’s seeemingly endless desire for more and more pictures of kittens doing cute things. The better you arer at deploying skateboarding cats, the better you are at devops. Simple as that.
A stands for Agile
Devops was invented because sysadmins felt they were missing out on the whole agile party. So devops is just agile plus sysadmins. AMIRITE?
L stands for Letters
Acronyms are nothing without letters. Letters are very much the key ingredient of a good acronym. The trouble with acronyms though is that the letters don’t always lend themselves to a word that’s relevant to your topic. One way around this is to think of a world beginning with that troublesome letter, let’s take the letter L for example, and let’s randomly pick the word “Lean”, and then simply write a book which demonstrates how “Lean” is actually quite relevant and applicable to the topic of devops. It’s a clever one this, because you can then use it to help flog copies of your book.
M stands for Money
Doing devops will make you rich beyond your wildest dreams. A recent survey has discovered that firms with high performing IT functions are less likely to suck ass than one’s with crappy IT teams. It only takes a medium sized leap of faith to believe that this has anything to do with devops. So it’s crystal clear then – doing devops means your organisation will outperform your competitors and we’ll all be sipping cocktails on a beach somewhere by this time next week.
So what are you waiting for? Go deploy those skateboarding cats and I’ll see you on the beach!
London Puppet User Group Meetup
London, Thursday May 21st, 2015
DevOps Exchange London – DevOps & DevOps
London, Tuesday May 26th, 2015
London Agile Discussion Group – Should DevOps be a person or a team-wide skill?
London, Tuesday May 26th, 2015
AWS User Group UK – meetup #15
London, Wed May 27th, 2015
Chef Users London – Microsoft Azure / Chef Taster Day
London, Friday May 29, 2015
9:00am to 5:00pm
DevOps Cardiff – Herding ELKs with consul.io
Cardiff, Wednesday, June 3, 2015
Agile Testing – Visual Creativity: Using Sketchnotes & Mindmaps to aid testing @ #ltgworkshops
London, Thursday June 4th, 2015
ABC (Agile Book Club) London – Review Jeff Patton’s User Story Mapping
London, Thursday June 4th, 2015
Agile Testing – Hooking Docker Into Selenium @ #ltgworkshops
London, Thursday June 4th, 2015
UK Azure User Group – Cloud Gaming Hackathon
London, Saturday June 6th, 2015
London DevOps – London DevOps Meetup #10
London, Thursday June 11th, 2015
Kanban Coaching Exchange – Continuous learning through communities of practice – Emily Webber
London, Thursday June 11th, 2015
Lean Agile Manchester
Manchester, Wednesday June 17th, 2015
London Lean Coffee – Holborn
London, Thursday, June 18th, 2015
UK Azure User Group – Chris Risner
London, Thursday June 18th, 2015
Jenkins User Conference – Europe (London)
London, Tuesday June 23rd – 24th, 2015
BDD London June Meetup
London, Thursday June 25th, 2015
Automated Database Deployment (Workshop – £300)
Belfast, Northern Ireland, Friday June 26th, 2015
1 day course
Database Continuous Integration (Workshop – £300)
London, July 8th, 2015
1 day course
Database Source Control (Workshop – £100)
London, July 8th, 2015
1 day course
London Lean Coffee – Holborn
London, Thursday, July 16, 2015
Agile Taster – a free introductory Agile training course
Cardiff, Saturday 18 July 2015
10am – 3pm
AWS User Group UK – meetup #16
London, Wed July 22nd, 2015
One of the questions I commonly get asked at the workshops we run is “What’s the connection between Agile and Devops?” or vice-versa. My usual tactic is to let the workshop attendees argue that one out amongst themselves. Firstly, because it’s quite an emotive question, so it gets people chatting, and secondly because I’m always keen to hear what other people’s opinions and experiences are.
Here are a few of the things I’ve heard from the workshop groups:
- “Devops is an evolution of Agile”
- “Devops is the missing link between agile development and operations”
- “Devops is a subset of Agile”
- “Agile is a subset of Devops”
- “What time are we having lunch?”
I’ve heard strong arguments in support of each one, and I’ve also heard people say there is no link between agile and devops (again, with a strong argument to support this opinion).
For me though, I like to think of devops and agile as complimentary. I also think devops is a key enabler for enterprise agility, and this is where things get a little murky. I don’t want to get into any great yak shaving exercise, but before we can describe the relationship between agile and devops, we really need to define what we mean by “agile”.
Agile development vs Organisational Agility
Agile development comes with many different frameworks, some more prescriptive than others, but they all share a certain number of core “agile“ features in common – namely that they embrace change, and are able to comfortably respond to it (ok there are plenty of other features in common, such as focus on customer value and working software etc, but I’m going to concentrate on the responsiveness to change for now).
Scrum, on the other hand, is a framework for helping you do agile software development, it has clearly defined team roles and a few rules you need to follow. But let’s be clear – Scrum is NOT agile. Don’t confuse the two!
Scrum is a framework for helping you do agile software development, doing agile software development helps you become agile
Now let’s take a deeper look in to what being “agile” is all about. It’s about being able to respond to changing conditions, changing market forces, changing requirements, and not only survive, but to succeed. “Change” is the arena in which agility thrives. So, to be agile you need to be able to react and respond to changes quickly. This is where the agile world and devops come together. Devops encourages closer and more meaningful collaboration between the business, development and operations, to help organisations deliver higher quality applications that can be deployed quicker, be maintained and monitored more easily, and provide fast and accurate feedback to the business. Without these key enablers, you’d be hard pressed to respond or react quickly to anything, let alone changing requirements, changing markets and changing technologies.
So in a nutshell, agility is all about the ability to embrace change, and succeed in a changing environment. Devops is one of the key ingredients to helping you achieve this.
Can you do agile software development without doing devops? Yes, of course you can, you could do Scrum, for example. But doing agile software development and being agile are two very different things.
So, now for the harder question: Can you “be agile” without doing devops?
Some might argue that this must be possible, because organisations have been agile for longer than devops has been around – to which I would respond by saying that the term “devops” may be relatively new, but the ethos behind it has been around for as long as agile has.
I imagine that under certain circumstances it may be possible to be highly agile and yet have an anti-devops culture, where there’s very little collaboration between the business, development and operations, but I’ve not actually come across it, if it does exist. In my experience, highly agile organisations from start-ups through to large enterprises, are embracing the principles behind devops whether they know it or not.
In summary, agile is NOT about sprints, it’s NOT bout stand-ups, it’s NOT about retrospectives or any of the following:
You can’t do agile, you can be agile, but you can’t DO it. You can “do” scrum, XP, Devops, TDD, BDD etc – these are explicit activities, unlike agile, which is more of a set of values and principles. You can perform all of the individual activities listed above, independently of any other. Sure, they certainly complement each other, but they can be done separately. The same can be said of agile software development and devops.
The relationship between devops and organisational agility is that one is an enabler for the other. The relationship between devops and Scrum, for example, is simply that they are both enablers for organisational agility, but they can also exist independently of each other.