This is going to be a quick
rant post, hopefully. Today I saw another Kanban board which had a “Read for test” column on it, here’s the screenshot:
I Think “Ready For” Columns Are Baaaaad
With most Kanban boards you mark a card as done when it’s ready to be pulled into another column. If that means it has to be deployed before a card is ready for test then so be it. The last thing we want is cards just sitting around waiting – this is baaaaaad. “Ready for Test” usually means it’s either deployed (and yet to be tested) or waiting to be deployed. Either way, not much is happening to the work sitting in this column. Basically it’s waste (or “muda” as the Lean Kanban aficionados might call it), and remember, waste is baaaaad.
Seriously, I Think They’re Baaaaad
A result of using these “Ready For x” columns is that they tend to slightly move us away from the “stop the line” practice that good Lean/Kanban systems employ. Basically whenever there’s a problem, or a bottleneck is appearing, we want to stop the production line and address the issue. So, if we keep all these “Ready for QA” cards in our In Dev or Code Review Column (basically whatever column comes before your Ready for QA column) then we’ll very quickly reach our WIP (Work In Progress) limit and the line will be stopped. That’s a good thing! We want to catch that bottleneck as soon as we can, we don’t want to hide it by pushing our cards into another “buffer” column.
Did I Mention That I Think “Ready For” Columns in Kanban Are Baaaaaad?
Yet another problem with “Ready for x” columns is that they quite often tend to be push rather than pull columns. You can’t really pull into a Ready for QA column as it isn’t an actual “workflow” state, it’s a “wasteflow” state (see what I did there?). I mean, who’s going to pull stuff into that column anyway? I’ve yet to meet a “ready for test” team who just sit around pulling cards into their column before marking them as “ready” (presumably once they’ve verified that they are indeed ready for test). Ok, you might have a deployment team who are responsible for deploying stuff to your test environments and so forth. In this case, your workflow state still isn’t “Ready for test” it’s “In Deployment”.
“Ready for x” columns make baby Jesus cry.
SAFe (the Scaled Agile Framework, with a random “e” at the end) seems to be the talk of the agile world at the moment, and as you’d expect, opinions are both strong and divided. On the one side you’ve got the likes of Ken Schwaber and David Anderson (the guys who brought us Scrum and Kanban respectively), while on the other side you’ve got, well, pretty much anyone who stands to make any money out of SAFe.
I’m working at a company in London who might be about to go down the SAFe route, so obviously I’ve had to do a bit of research into this new framework, as it could soon be directly impacting me in my capacity as an agile coach and devops ninja.
My initial questions were:
- What the hell is this and how come I’ve never heard of it?
- Why is there no mention of devops?
- Is it really as prescriptive as it sounds?
- Isn’t it a bit “anti-agile”? (recall “individuals and interactions over processes and tools”, part of the agile manifesto)
So I started looking into SAFe. There’s quite a bit of information on the SAFe website, and the “Big Picture” on the homepage is crammed with information, jargon and stuff. At first glance it seemed fairly sensible on the higher levels, but overly prescriptive on the team level. Then you start clicking on icons, and that’s when it gets interesting…
Story Sizing, Velocity and Estimating: They tell us that all teams should have the same velocity and the same sizing (i.e. 1 point in team A should be exactly the same as 1 point in team B). They also say that 1 point should equal 1 day. I find this quite interesting, as this time-based evaluation is flawed for a number of reasons. Velocity (in normal scrum) should be obtained by measurement, not by simply saying “right, there’s 5 people in the team, 5×8=40, therefore our velocity will be 40!” This pains me, because I firmly believe that each team should be allowed to work out their own sustainable velocity, based on observation and results (and applying the Deming cycle of making a change and seeing if it improves output). If we are all given a goal of x points to achieve in a sprint, all we will do as a team is fiddle our estimations so that we hit that target. That’s exactly why we don’t use velocity as a target. What am I missing here?
ScrumXP: SAFe seems to suggest that you MUST do 2 week sprints. You have no option in this. Doesn’t seem to matter if you want to have a kanban system based around a weekly release schedule. SAFe seems largely ignorant of this. Is SAFe suggesting that Kanban doesn’t work? Has anyone told David Anderson?
SAFe prescribes Hardening Sprints: These are sprints set aside at the end of every release (one every 5th Sprint), to allow you to do such things as User Acceptance Testing and Performance Testing. In Continuous Delivery we work towards making these activities happen as early as possible in the release pipeline, in order to shorten the feedback loop. We really don’t want to be finding out that our product isn’t performant a day before we expect to release it! I certainly wouldn’t encourage the use of hardening sprints in the SAFe way, instead I would encourage people to build these activities into their pipelines as early as possible. I think of hardening Sprints as a bad smell, isn’t it just a way of confessing that you don’t expect to catch certain things until the end? So rather than try to fix that situation and reduce that feedback loop, you’re kind of just saying “hey, s**t happens, we’ll catch it in the hardening sprint”.
Innovation Sprints: These happen at the same time as the hardening sprint. SAFe is suggesting that during a normal sprint we don’t have time for innovation. And that is quite often the case – but wouldn’t it be better if we actually did have sufficient time for continuous innovation, rather than actually have a dedicated half-sprint for innovation? The book “Slack” by Tom DeMarco talks about the myth of total efficiency, and suggests that by slowing down and building in some slack time, we get greater returns. This is better achieved as part of everyday practice rather than working at some mythical “total efficiency” level and then having an “innovation sprint”. The SAFe approach seems to be an easy option. Rather than taking the time to determine a team’s sustainable velocity which includes sufficient time for innovation, it suggests just saving it up for a sprint at the end of every release. Don’t forget that at the same point, the team will apparently be doing “hardening” activities, gearing up for a release, and planning the next one. For some reason I feel uncomfortable with the idea that innovation is something that should be scheduled once every 10 weeks, rather than something that should be encouraged and nurtured as part of normal practice.
The Scrum Master: SAFe has this to say about the Scrum Master:
In SAFe, we assume that the Scrum Master is also a developer, tester, project manager or other skilled individual (though not the team’s manager) who fulfills his Scrum Master role on a part time basis.
Wow, that’s some assumption. They seem to suggest that you can just take any developer, tester etc and send them on a scrum course, and hey presto, you have a scrum master. And yes, you could do this, but what sort of scrum master are you getting? They also say:
responsibilities can generally be accomplished in about 25% of the time for that team member
which I again find surprising. A Scrum Master is just one quarter of a person’s time?? Seriously? Mentoring a team, coaching individuals, removing impediments, applying the principles of Scrum, helping the team work towards a goal, leading a team towards continuous improvement – all of these things are expected of the Scrum Master in SAFe, and yet they can all be achieved in “about 25%” of a person’s time, apparently. And where does an agile coach come into this? Well, they don’t exist in SAFe. In SAFe you have SAFe consultants instead.
The Product Manager and The product Owner: These are 2 very separate, very different roles in SAFe. A Product Owner works with the Scrum Team, but doesn’t have contact with the customer. The Product Manager has contact with the customer but deals with the scrum team through the Product Owner. Also the product owner doesn’t own the product vision – that responsibility belongs to the product manager – this seems strange to me, I would have naturally thought that the product “owner” would own the product vision. So essentially we’re adding yet another link in the chain between the customer and the team. I’m struggling to see this as a good thing, when in my experience a close relationship between the business and the team has always been of great benefit.
There is no Business Analyst role in SAFe, which I find quite interesting. This role seems to have been split out into the Product Owner and Product manager roles. For instance, the PO is meant to do the Just-In-Time analysis on the backlog stories.
in SAFe, the UX Designer is NOT part of the agile team. Rather, they work “at the Program Level” (whatever the hell that means, possibly on a different floor, maybe) yet they still do the following:
- Provide Agile Teams with the next increment of UI design, UX guidelines, and design elements in a just-in-time fashion
- Attend sprint planning, backlog grooming, iteration demos
- Work in an extremely incremental way
- Rely on fast and frequent feedback via rapid code implementation
- Are highly collaborative, and…
- The UI criteria are included in the “Definition of Done” and User Story acceptance criteria
But I must remind you that according to SAFe they are NOT part of the agile team :-) Is it just me or does this come across as a bit, I don’t know, pedantic?
Pretty much the same rule applies to devops (which was included in a later version of SAFe) – devops people aren’t in the team BUT, you can simply achieve “devops” in part by:
integrating personnel from the operations team with the Agile teams
Er, ok. So they’re not part of the team but they’re integrated with the team. Riiiiight. On a plus note – it does mention “designing for deployability”, which can never be overstated in my opinion.
These are just my initial observations and I’m sure I’ll have a lot more to say on the subject as we embark on our SAFe journey. I’m hoping it’s not as prescriptive as it sounds, as I honestly don’t believe there’s a one-size-fits-all solution to adopting Agile. I very much believe that every organisation needs to go on their own journey with agile, and find out what works best for them. It’s my opinion that the lessons you learn on this journey are more important than the end result. In my experience, most organisations will have invariably witnessed a fantastic cultural shift during their gradual transition to agile, and I find it very difficult to see how a prescribed framework such as SAFe can facilitate this cultural shift.
The Agile Silver Bullet
So is SAFe really an agile silver bullet? I doubt it, but time will tell. I certainly don’t disagree with the majority of the contents of the “Big Picture” but where I do disagree, I feel very concerned, as I seem to disagree on a very fundamental level.
I would be much happier if SAFe was a lot less prescriptive-sounding. I can see SAFe being popular with larger-scale organisations with a penchant for job-titles and an unhealthy affinity for bureaucracy, I mean, it’s a framework, and they lap that stuff up! I can also see it being quite effective in those situations, after all, pretty much anything’s better than Waterfall!
I can see SAFe appealing to people who aren’t prepared to go on the agile journey, because they fear it. They fear they will fail, and they fear a lack of clarity. This framework puts nice titles everywhere, tickboxes to be ticked and nice clear processes to blindly follow. I can imagine it would be hard not to look like you’re making progress! I don’t yet trust the framework, but that could still change, but for the time being I’ve got the impression that it’s command-and-control agile, more of a tick-box exercise than a vessel for personal and organisational development.
Ken Schwaber and Jeff Sutherland, the original guys who came up with the whole concept of scrum back in about 1995 have recently posted a video on the interwebs, explaining some changes to the scrum model based on their experiences over the last few years. The video can be found here.
If you don’t have the time to watch the video, here’s my summary of the bits I found most interesting:
1. We should do more prep before our sprint planning, so that all stories are sufficiently prepared before the sprint planning session. This has come about because many sprint planning sessions take many hours. They have suggested having a “ready” status for backlog items that are ready to be discussed in the planning session.
2. We should always have a sprint goal, and during our daily stand-ups we should talk about how we are helping the team progress towards our sprint goal
3. We should talk about “value” in our sprint reviews. With hindsight, did we deliver as much value as we could have? If not, what could we do next time to ensure we deliver greater value?
I started running scrum of scrums sessions a few months ago where I work, with the intention of getting the teams to collaborate and communicate effectively, particularly in relation to any areas of overlap between the teams. There are quite a few teams who work on the same project/solution, and so their effective communication is essential.
As it turns out, in most cases the comms between the teams was good, and these meetings ended up being more of a knowledge sharing session. We discussed any new ideas, tools, practices which we’ve been using, and whether they’ve worked or not. We also discuss what isn’t going so well for each team and we collectively make useful suggestions and try to help out.
So basically it’s not so much a “scrum of scrums”, but more like a “retrospective of retrospectives” :-)
Bertrand Russell once gave a lecture on astronomy, and how the earth rotates around the sun, and how the solar system is part of a vast galaxy, etc and so forth… At one point he was interrupted by an old lady, or so the story goes, who said “What you’re saying is rubbish, the world is a flat plate resting on the back of a giant turtle”. When Bertrand Russell questioned her as to what’s below the turtle, she replied “Another turtle”. Bertrand persisted with “And what’s below that turtle then?” to which the lady replied “It’s no use – it’s turtles all the way down!”
I was reminded of this story recently when I was in discussion with a colleague about sprint planning. My colleague was explaining to me that they use story points for sizing and estimating stories, but they then break their sprint stories down into smaller tasks and use hours to estimate these tasks, rather than continue to use points.
I had never come across this particular practice before. Of course, I’ve seen teams who do everything in hours, I’ve just never worked with a team which used both points and hours. I’ve always worked in points for story sizing, and when the stories need breaking down, I would break them down into smaller points: It’s points all the way down! I’ve never had any problems with this approach, so I was curious as to the advantage of switching to hours.
My first port of call, as always, was Google. When that came up empty I turned to Twitter, and got a friendly response from none other than Mike Cohn.
— Mike Cohn (@mikewcohn) July 19, 2013
Now, I never thought I’d find myself disagreeing with Mike Cohn, indeed his book “Succeeding With Agile” has been something of a bible for me for the last few years, but on this issue I am still not convinced.
Mike has written a really interesting blog post on why he uses hours rather than story points for sprint planning (you can find it here). One of his reasons is because he feels hours are better short-term measure than points. In some circumstances I would agree with him (perhaps if your stories are MASSIVE, and you need to break them down into much smaller tasks), but in my experience it doesn’t generally appear to hold true. I use points for both stories and tasks – the only difference is that when we get around to task estimation & sizing, we expect to be a little more accurate than when we estimated the stories in the product backlog. Quite often we will take a story of say 5 points, and break that down into two 3 point tasks. So 3 + 3 doesn’t add up to 5. Who cares? the 5 was an estimate anyway, and I always push to make everyone understand that estimates are just guesses, and the further out we make the estimate, the more approximate it’s going to be. By the time we get around to doing our sprint planning i expect us to take a much deeper look into these stories, and benefit from additional analysis, so naturally I expect our estimate to change.
I usually do everything I can to steer away from hours, because it can often be too confusing for people outside of the team to understand why the total hours don’t add up to the total available resource hours. I’d rather just not have this conversation sometimes! Also, after explaining to everyone the key benefits and purpose of using points, I can imagine they’d be confused as to why we were suddenly ditching all the advantages of points, and switching back to hours for the sprint planning. By the way, I’ve covered the reason why we use points over hours in my previous blog post here.
Anyway, I just thought I’d write this post to show that both approaches are common, and I guess it’s just a matter of “whatever works best for you”. But I’m still not convinced with the whole hours thing!
FUN With Points!
Over the last few months I’ve been getting some teams up and running with Scrum, and one area I always seem to have “fun” with is introducing people to the points-based system of estimating. So much fun in fact, that I’ve decided to write this very blog post about it! See how much fun it is already? No? ok.
When we do our estimating (during the planning sessions) we look at each story (work item or task) and guess how big it is in terms of effort. But we don’t just think about how long it’ll take, we also consider the following:
- Amount of unknowns
So for our teams, the size of a story represents how much effort it’ll take, how complicated it is, and how unsure we are that we know everything we need to know about getting that task done.
But Why Not Just Use Hours?
I get this question quite a lot. There’s a couple of reasons why hours don’t rock my world in quite the way that points do. Firstly, they’re an absolute measurement, rather than a relative measurement (more on relative measurements in a bit), and also because they encourage us to focus on how long something will take, and discourage us from thinking about complexity and unknowns. When estimating in hours, we naturally think primarily about the actual amount of time it would take to do that task, and hey presto, we have an answer. This answer quite often doesn’t take complexity and “unknowns” into account – but why? Basically it’s because we’re human. If were asked to think in hours, we naturally just think about how long a task will take, not how difficult it is or how many unknowns we should consider.
So when we’re all finally agreed that hours just doesn’t cut it, I go straight into the whole “points” concept. Unfortunately, for some people this basic concept is far too similar to hours, and they get confused. I was talking about this with Matthew Skelton (of London Continuous Delivery group fame – aka @matthewpskelton), and he suggested that I start off by getting people to use completely abstract objects to do their sizing, such as dinosaurs for example. I think this is a great idea for an exercise to get people away from thinking in terms of hours, but I’m not sure how well it would work in an actual sprint – you might need a team of people who are dinosaur fanatics! Anyway, I’ve mentioned this dinosaur idea a few times, in order to demonstrate that our points are completely abstract and not directly related to hours, and it has worked nicely (cheers Matthew!).
Next we move on to the exciting topic of relative sizing. Once the team have got their heads around the idea that a story shouldn’t be estimated in hours alone, it’s time to teach them a new way of measuring how big a story is.
There was a paper published in the American Journal of Psychology in 1967 called “Size-Estimation of Familiar Objects under Informative and Reduced Conditions of Viewing” by H. Richard Schiffman (vol 80 pp229-235 for those that are interested) which revealed that humans are better at estimating the size of an object when they had another familiar object to compare it to. We use this method of “relative sizing” when we estimate the sizes of our stories or tasks. We start off by picking the easiest (and usually the smallest) story of all, and give it a number – if we think this really will be just about the smallest sized story we ever want to record, we can give it the size of 1. I call this our “starter”. Some teams have given their starter a score of 2, because they often get stories which are much simpler, but not that frequently. I have no problem with this idea.
After we’ve defined our starter, we size all our other stories in comparison with it. So if our starter is 1 point, and our next story is 3 times bigger than our starter, we give it a 3! Simples.
Some people use the Fibonacci Sequence as an additional rule in their estimating. The Fibonaccci sequence goes 0, 1, 1, 2, 3, 5, 8, 13, 21…. etc and so forth. There aren’t many things more boring in life than people who are obsessed with the Fibonacci sequence. The only interesting thing about it is that male bees ancestry follows the Fibonacci sequence (has 1 parent because he came from an unmated female, 2 grandparents, 3 great-grandparents, 5 great-great-grandparents, and so on). Bees aside, the Fibonacci Sequence is fairly dull. One reason why some people use it in agile estimating is because they argue that if a story is bigger than a 3, then the sheer size alone should account for an additional increase due to the potential “uncertainty” factor. Suffice to say, I don’t agree.
I read a book called The Agile Samurai by Jonathan Rasmusson (I really should do a book review of that at some point) and I came across a section explaining “The Wisdom of Crowds”. The Wisdom of Crowds, it explains, happens to be another book (by James Surowiecki) which, bear with me here, tells a story about a British scientist called Francis Galton. In 1906 Francis Galton did an experiment at a fair where people had to guess the weight of a butchered ox. Francis expected a professional butcher to provide the most accurate estimate, but to his surprise, the crowd of villagers actually came up with the best guess! In short, experts are often trumped by a crowd.
This concept has been adopted in agile estimation, where we use group consensus to help us agree what our estimate should be for a particular story.
That’s about it from me when it comes to points. So remember, hours are bad, dinosaurs are good, butchers are stupid and male bees have no fathers!!
If Devopsdays London was an office party, Patrick Dubois would be the Head of HR keeping an eye on proceedings, the guest speakers would be the live entertainment, and the Zero Turnaround guys would lacing the punch with more vodka and photocopying their body parts.
Zero Turnaround bought the fun, as evidenced by their stand which looked more like a booze section from an off licence, and this video here, which had me giggling in the back row like a naughty school kid:
Watch out Grammy Awards. Naturally, I felt I had to go and have a chat with these guys during one of the breaks and find out if I could blag any of their free booze…
“It’s all been taken, by competition winners” explained Simon from behind a massive heap of Guinness and Irish Whiskey. “We’re just, er, looking after this lot” said Neeme (Simon’s Zero Turnaround partner in crime) Yeah, sure.
No luck with the free booze, I thought maybe I could distract them with a question or two about devops, and then pinch some beer while they were deep in thought.
“So what does DevOps mean to you guys?” I asked, with half an eye on a bottle of Jamesons…
“A lot of it is about streamlining and bringing value to the customers sooner” Said Simon Maple, Tech Evangelist at ZT. And I suppose that’s actually what it’s ultimately about. Bridging the Dev and Ops gap is really about making us work as a team more smoothly, which leads to us being able to get stuff to our customers more quickly and more reliably. “But of course you need to right tools to do it – you have to automate across Dev and Ops” He added “This helps us remove bottlenecks”, which reminded me about my bottle of whiskey objective, so I asked another question by means of a diversion…
“Do you see a big crossover between DevOps and Agile” I asked. “Yes, DevOps is sort of like Agile breaking out of Dev and into Ops, but you can do devops stuff without being Agile” which again is true, but I imagine if you’re truly agile as a business, you’re almost certainly going to be doing “devops”.
Next, I decided to chuck in a question that caused a whole heap of opinionated discussion at devopsdays, namely “Is DevOps a job title?” (correct answer of course is – “what does it matter what your job title is? You can call yourself the Prince of Darkness for all I care. What matters is getting stuff done”), but Simon went for “I don’t think so, is Agile a job title?” which also made a lot of sense.
“So” I asked, “what do you see as the main challenges to DevOps?” I thought this questions was sufficiently fluffy to warrant an equally fluffy response, during which I could maybe accidentally knock one of the cans of Guinness into my bag…
“Well DevOps has well and truly gone viral now, so our main challenge is to make sure we do it right, or it’ll get a bad name” said Simon, as if he’d heard that particular question a million times before. No luck with the Guinness there then.
After about 5 minutes of chatting I was no closer to bagging myself any free beer than I was when I turned up, but I was having a good chat, and eventually we got around to talking about their products, JRebel and (the one I was more interested in) Live Rebel. I’d already seen their demo and it looked pretty cool to me. I’ll trial JRebel soon and put a review up here too, as I think people will be pretty impressed (I know I was), but that’s for another time. If you want to know more about their products, go see their website, duuh! For those too lazy to go to google and type in “Zero Turnaround”, here’s their website: http://zeroturnaround.com
For those even lazier than that, here’s a REALLY brief summary:
Live Rebel is a deployment orchestrator. It rolls up code and db changes into a release package and automates the delivery to whatever environment you like. It supports Java, Ruby, Python, PHP and Perl (officially) and Neeme also says it’ll work with .Net as well (not so officially). It also plugs nicely into most decent CI systems, such as Jenkins/Hudson and Bamboo (for which there are plugins) and has a one-stop management console for managing your servers and environments etc and so forth.
Anyway, I still had one final question for Simon and Neeme… “What has been your highlight of devopsdays then guys?” I asked. “Meeting the real creme de la creme of the industry and just being part of a great conference” came the reply. So, not meeting me then. Thanks guys. :-(
Simon Maple is @sjmaple on twitter and Neeme is @nemecec. Follow them at your own peril.