This automatically-generated transcript is taken from the IT Pro Podcast episode ‘Stamping out scope creep'. To listen to the full episode, click here. We apologise for any errors.
Hugh Armitage
Hi, I'm Hugh Armitage.
Adam Shepherd
And I'm Adam Shepherd.
Hugh
And you're listening to the IT Pro Podcast where this week we're channelling Thom Yorke and talking about creep.
Adam
There are any number of potential pitfalls associated with running a live development project within an organisation. But one of the hazards that frequently ends up causing problems for dev teams is the dreaded spectre of scope creep.
Hugh
As projects progress, it's easy for their remit to gradually morph and expand, eventually growing to the point of becoming unwieldy and impossible to manage. Controlling and limiting scope creep is a finely honed craft, and one that the best project managers have perfected, because without keeping a close eye on it, unrestrained creep can have major negative impacts on an organization's development strategy.
Adam
Joining us this week to talk about the dangers of scope creep, as well as some of the strategies for preventing it is Jess Cregg, developer advocate at LaunchDarkly. Jess, welcome back to the show.
Jess Cregg
Thanks so much for having me.
Adam
So, Jess, you've worked on a number of projects. To start us off with, can you give us some examples of scope creep in action?
Jess
Yeah, absolutely. So scope creep. It's, it's, it's a very human problem. And I was chatting to a lot of people about it recently, I found one person put it really well. They said that scope creep often happens when you confuse your project team, with your product team's aims, it can often happen when you're designing for the problems that you're going to have in the future, as opposed to the problems that you're facing right now. So one of my favourite instances was I was chatting to a speaker that I met at a conference recently, and he was talking about how he was once working on a team where they were creating a UI and a database for changing the country value in their product, a value that could have been tweaked with three lines of code and was probably going to be changed once every three years. So that's thousands of pounds for something that gets used once. It's maddening.
Adam
And that seems like it should be, as you said, a fairly simple problem to address and one that really doesn't need that much engineering, you'd think.
Jess
Absolutely. But it's, it's one of those problems that's hardwired into us, we experience the pain once of something going horribly wrong. And we know the worst case consequences. So we try to avoid those in a very helpful human evolutionary science. Scope creep can come really, it can really easily infiltrate your team. And one of the ways that, one of the one of my favourite ways of preventing it from happening is to keep your teams really small. Everyone's talked about the, everyone speaks about the the two pizza rule. Have you heard about that?
Adam
Yes. It's one of my favourite rules.
Hugh
I'll have two pizzas, please.
Jess
So the two pizza rule is a rule that was made really famous at Amazon. It's another FAANG concept that's taken the entire industry by storm, essentially, it's the idea that your team should only be big enough so that they can be fed by two large pizzas.
Adam
Yeah, it's a good rule, but slightly flawed in that I know many workplaces where that would result in a development team of one and a half people.
Hugh
I was gonna say two, but one and a half. Yes.
Jess
I mean, it's not only great for keeping team delivery orders down, but for preventing teams from growing. But when I was recently reading the DevOps Handbook they're talking about one of the main reasons why this is implemented is, is due to, we have a hard limit of the number of relationships that we can form with people.
Adam
This isn't something I'm familiar with.
Jess
So it's the phenomenon that as humans, usually we can form quality relationships with about 150 people.
Hugh
It's like the policing thing, how you can control a small town is fine, and then you get more people and then you get chaos.
Jess
Exactly. And when those sort of chaotic relationships start to breed, we can kind of start to misinterpret where the boundaries lie between our commitments to a project. When we're faced with a problem, we think more heads equals faster solution, right? But that can only be true when when work is completely interchangeable, when you can just throw a bunch of people onto a project and they can get on with it without without any interaction. My favourite analogy for this is, is I don't know whether on not you used to play Age of Empires?
Adam
Oh, yeah.
Jess
But whenever you used to throw just as many villagers as possible on a build project, they could just get on with it, and it was absolutely fine. But it's just not really the case with software, unfortunately. You need to be able to speak between each other, you need handoffs for different subtasks, and it's actually been proven that the more people that you throw on a software team, the later the project becomes, so you not only end up compounding things in terms of time, but then that puts everyone under pressure and they're pressured to deliver more over their agreed aims.
Hugh
It sounds a bit like a sort of personnel problem, like we're hardwired to devolve to scope creep, if we're not guarding against that.
Jess
Definitely, I mean, we want to, we don't really want to meet the aims, we want to be able to like exceed what we're going for in our team.
Adam
So one of the one of the best kind of illustrations of the principles behind scope creep, for me, is, when somebody leaves a business, often in the interim, before a replacement is hired, one of their colleagues will end up assuming some or all of their responsibilities in the kind of in the interim. And in a lot of cases, that just becomes the norm, you know, the hiring process goes on, maybe it's a couple of months, or, you know, even longer before a replacement for that person who's left can be hired. And over the course of that period, those kind of temporarily assumed responsibilities, just become part of that person's day job. And that is, I think, a really good illustration of how easy it is for scope creep to creep in, if you like, to, basically, anything, and it's especially the case in software projects, I think, particularly when you have large numbers of business stakeholders who aren't actively involved in the development process, and don't necessarily understand that, you know, when they say like, oh, maybe we could add such and such, you know, capability, or, you know, maybe it would be good if it could also cover, you know, this business area, or geographic or whatever, isn't just a case of tweaking a couple of variables.
Jess
That's the version of scope creep, I've become the most, that I've had the most personal experience of, and it's really easy for it to happen, because as soon as it's just one more thing, things can spiral out of control. And the just one more thing rule doesn't usually end up eating into the main sort of time that you frame for your project, for the thing that you're developing, it falls into the sort of, into the the time you've allocated to either cut down on technical debt, or the time you've allocated to perfect your overall process, you know, that 20% time that we talk about in DevOps. I've seen a lot of, a lot of companies recently, they've been organising their teams differently to stop scope creep. And I think this is kind of a bit of a nod to Conway's Law, which is the idea that - have you heard of Conway's Law?
Adam
I have not.
Jess
Okay. It's this idea that we produce, the things that we produce are indelibly impacted by the way that we're organised. Anyway, it was studied by Professor Conway. And he found that he tasked two teams to produce two different compilers, and five people working on one project, I think, a COBOL project and three people working on an ALGOL project. And it turns out the finished state compilers each had three or five stages, depending on how many people were working on that product. So it shows that, we see this all the time in products that we use, usually, the way that we're organised if you have, if you have security experts on your team, you usually work in a different way. And it's can be, it's I mean, it's often to our detriment, because we often end up working with the people that we have, you know, the skills crisis, you're kind of, you have as many people as you can, you can get your hands on. But by kind of building teams really specifically and organising people into different squads and giving squads real specific remits on how, which work they're allowed to factor into their daily processes, you kind of are able to sort of clamp down on scope creep a little bit.
Hugh
What is it about scope creep, that makes it damaging?
Jess
It's really damaging for a few problems, it can really, it infiltrates the way that your, your order of processes, so it can circumvent your established way of doing things, which means that it just interrupts your team from being autonomous. If you if people know where their information is coming from, then they know where to direct their efforts. And as soon as things change, it throws... if this is different, why, is this potentially different? You start to distrust the entire system. There's like four main models that people can use to like measure organisational health, and one of them is deployment frequency and you only really, you're only really going to deploy if you're really like confident in what you're deploying, if you're confident in your product. So it shakes your confidence in what you're doing, but also shakes your confidence in the overall way of doing things. So there's the idea of a microservices architecture, it only works because you're designing for things that can fit into your head, that's what the author of Team Topologies said, you've got to design for software that can fit inside your head. So as soon as things start to become complicated by one or two degrees, quickly gets amplified, it also gets amplified because it's, you're communicating a different thing through many different people. And it's got the, you've got the added risk factor in there, right?
Adam
Yeah, absolutely. I think that one about designing software that can fit in people's heads, I think is super important. Because when you are dealing with particularly kind of business systems, as soon as you start needing complex flowcharts, and diagrams, to explain kind of how your app works, or how your software works, and what it does, it becomes much much easier for bits to fall through the cracks. And to end up with this Kafkaesque nightmare of code where, you know, there's 18 different modules, no one's entirely sure which specific functionalities each one governs. And then kind of that's, that's the stage where, you know, when one element breaks, the entire thing has to be forensically broken down and troubleshot to find out, you know, what's wrong, and how to fix it, which - and again, it goes back to the question of technical debt, I think - the larger you allow scope creep to become the greater risk it introduces of unfeasible technical debt build up.
Jess
But we're talking about scope creep like it's, I mean, it is, it is a bad thing in a lot of instances, but you kind of brought up the the idea of incidents there and scope creep can be really powerful from an incidents perspective. I mean, if we're looking at game days and chaos engineering, the idea of creating incident days where things go wrong in a way that's outside of your scope to correct, is a fascinating way of finding out just how things fit together. So I think that's like a really good way that scope creep can be used to, to kind of strengthen your organisation.
Hugh
That's a really interesting confluence, because scope creep coming out of a place of people being worried about making errors and you know, we've got to account for everything and then, but the positive side of it being creating errors and creating chaos that could be beneficial, rather than just negative.
Adam
Are there any other benefits to kind of permitting a certain amount of scope creep within your kind of development environment or project management environment?
Jess
I think that the main one seems to be stress testing your organisation, the main thing seems to be that if you permit a little bit of scope creep, scope creep, you're able to see whether or not people can, your engineers can rise to the challenge a little bit. But the only issue is, it's it's such a good idea on paper, but it's just often really, it's often carried out in bad faith. We're not in the situation yet where by rule, organisations tend to vouch for their people. And when that is the standard, then sure, we can use scope creep to stretch ourselves and to challenge ourselves. If you permit it within a set period of time, I mean, for instance, like hackathons are all scope creep, they're like a scope creep challenge to see how far you can take the bounds of something.
Adam
So we've talked about some of the ways that you can, if you like, deliberately introduce certain elements of scope creep, or a certain level of scope creep into your organisation, things like hackathons and chaos engineering, and all that kind of stuff. What are some of the main ways that scope creep comes into projects unintentionally, do you think?
Jess
So okay, so scope creep always comes into projects unintended and unintentionally when organisations are either heavily weighted towards the revenue side of the org and that happens all the time, right, because salespeople are speaking to customers all of the time, so they know exactly what they want, but also salespeople are wired in a way that they work in a very explicit manner, which is excellent, but also in very immediate terms. So hearing about a potential feature can sometimes become the difference between a deal happening and a deal not happening. And then sudden, suddenly you're, you're running towards a product with really disparate aims and no main strategy, that's often when you start to drift away from the idea, or the idea of your customer profile, right? You start designing for this person that doesn't exist, it's like the Doctor Who problem; you're demographically kind of designing something for an audience that doesn't quite... I mean, is, do people actually use that feature? And is that just this small organisation over here? That's why positioning the product team and making sure that the product and product management team is able to be essentially a bit of a fortress against these customer success aims is, is really, really, really important.
Adam
Yeah, and one of the growing categories that we've seen over the last several years really growing in prominence, is that product owner, product manager role, which is essentially kind of geared entirely around around that, around nailing down, right, what is it that we need this particular product within our organisation to do? What aims is it meeting? What, you know, what is the roadmap for developing it? And where do we need to essentially cut off development and say, No, that is actually not necessary for this, or not feasible.
Jess
Yes. One of the, one of the really encouraging things we're seeing at the moment is that a lot of - I've seen this loads on Twitter - it's a lot of senior engineers are taking, their like developing this sort of sense, this sort of this reflex to understand Wait a second, is this, does, do do we actually need to build this? I remember seeing someone recently talk about how being a senior engineer is essentially having endless conversations about why you shouldn't make something.
Adam
Yeah, exactly. It's it's the whole, you know, you were too busy thinking about whether or not you could, you didn't stop to think about whether or not you should, from famous DevOps pioneer, Dr. Ian Malcolm. Like, yeah, I think, yeah, I think that's a really, really shrewd observation, a lot of a lot of being a senior technologist in general, and a senior developer in particular, is knowing when not to do something.
Jess
And it can often be really counterintuitive, right? Because if you are, if you're an engineer on a team, and you really want to step up into into, you really want to develop your career, you often need to find your niche and able to, to be able to kind of carve out your unique path, you know, you want to be the change management person, you want to be the, you want to be the person that understands feature flagging. For you to be able to do that you need to be able to carve yourself out as a specialist. But for the general overall team to thrive, you need to still be able to be like a generalist, essentially, like you need to be able to build a team where you can hedge your bets on their knowledge, and you can bet, okay, I believe that this team will be able to encounter this area of development that they haven't had any experience in because of these sort of things that they've experienced in the past. So it's very easy for you to take scope creep and see it as an opportunity to develop not only your career, but the kind of the, the reputation of your team. But that's that's I mean, that's, that's ego speaking. That's ego speaking, and that's only going to be successful for the short term. It's not a sustainable way forward.
Adam
Yeah, everyone wants to, to stand out and to be the, you know, the team or the staff member that goes Yep, we can, we can make that happen. We can deliver on these things that that you want to do. And also it's you know, in a lot of businesses, particularly ones where the development team aren't kind of, aren't necessarily the central pillar of the business, it can be hard to say, to say no to business stakeholders who go right we want such and such functionality or you know, we want this project to deal with X, Y and Z. It takes a lot of soft skills, ability and confidence to be able to push back on that and go, Well, no, actually, this isn't going to be a feasible project or this isn't going to deliver on what you want it to do for these reasons, and to have that be, you know, received and understood and acted on by, by business stakeholders. It's a big ask.
Hugh
That's a universal problem really, isn't it, just not in dev work but in all sorts of, you know, every areas of business, it's a problem we deal with.
Jess
Yeah, it really is. I mean, it's it's with with, you know, within the within the context of a development team, you can, you can end up wasting thousands of pounds, hours, it can become an expensive headache, but we're taught to say yes, as a way of falling in with the, the people that we want to see be seen as allies to. And not to blow this up too much. I mean, I've become a bit obsessed with, with with tech failures recently, and I find it amazing how these things flow , like Juicero or even Theranos.
Adam
Beautiful.
Jess
It's incredible stuff. But people say yes, when they really should say no, and you don't really want to be seen as a downer, you don't really want to be seen as a blocker. But in essence, you need to be able to all unite around the same idea. And that sometimes requires you saying no, right?
Adam
Yeah, I mean, it's it's organisational peer pressure.
Hugh
I think it's still quite hard, though, because it's not like we all work in institutions where that isn't frowned upon. And, you know, you have to, you have to have higher ups who are willing to hear no. And I mean, and whether that's true or not, there's there's often a feeling that you can't say No. And, you know, you need to combat that.
Adam
Yeah, I think a lot of that, as well feeds back into, you know, company culture and company environment, and how, how much are you listening to your project teams and your dev teams, and, you know, just your kind of junior staff in general, you know, because if you're, if as if, as a business, your leadership are not willing to actually sit down and listen, properly listen, to what your junior staff members are telling you, then yeah, that's gonna be a problem.
Jess
These sorts of bad decisions, they thrive in silence, they, the the best people on our team are usually the ones that really want to avoid context switching, they usually are the most adverse to communication and communication could sometimes be the most intense, like time intensive training activity that we need to, that they need to engage in. So I remember reading a study recently was talking about how collective intelligence in teams, you know, the ability to be able to do something well, on a collective scale, it thrives in teams that often spend fewer words communicating with each other, the longer they spend with each other, the more they seem to understand their nonverbal cues, and whether or not that's virtually or in person, they start to understand their innate cues around something. So this is why observability has become such an important part of preventing things going wrong.
Adam
So let's quickly address the differences between scope creep and feature creep. Because both are brought up fairly often in, you know, the development world. But there there is a distinction there that's that's worth noting. So what's what's the kind of the key difference for you?
Jess
Scope creep can happen to anything; scope creep can happen to a product, a project, it can happen, it's that additional few pieces of functionality, whereas feature creep, feature creep happens when a feature diverts from its original intended purpose. It becomes, it becomes a, like, an improv game; a 'yes and' situation.
Adam
That's a marvellous way of looking at it.
Hugh
I mean, in this case, is feature creep a sort of a species of scope creep?
Jess
Yeah, scope creep is more the, is the overarching term. And feature creep is, is the additional overload og features again and again, in a sort of a continuous stream. So it's...
Adam
It's like a subset of scope creep. I think it's the the phenomenon that non developers are probably most likely to be familiar with, in in like a software context. You know, we're all familiar with social media platforms are the ones that springs to mind first for me in terms of feature creep. We're all familiar with, you know, companies like Facebook and Twitter, bolting seemingly endless additional capabilities onto their core platforms, most of which, you know, no one really asked for. And it just, it seems in a lot of cases like adding new functionality for the sake of having new functionality rather than for what it actually offers in terms of benefits.
Hugh
Twitter stories and LinkedIn stories.
Adam
Exactly. LinkedIn stories - that much demanded and widely used feature.
Hugh
I'm sat at my desk, I love working, it's great.
Adam
Definitely isn't going to end up being quietly dropped in 18 months time.
Hugh
That's interesting, because feature creep does, as you say, kind of like that's quite easy to conceive. But scope creep is kind of, the more you think about it, the more vast it feels, and the more different ways it can come in. And different ways that it can cause chaos.
Jess
Feature creep is usually influenced directly by commercial pressures. So I don't know whether or not you've noticed this, whether or not, I think it's more of a sort of a thing if you're, if you use an iPhone, but your notifications are increasingly being used as sales, collateral or marketing, marketing collateral, and that's not necessarily what they're for. And you can kind of see that this has been led by the top down, even when you could jump into the settings feature, you can see sometimes a try this feature for x pounds per month. And that's it's that's the that's where you kind of see the commercial pressures of elsewhere in the business being leaked down into the into places where we shouldn't frankly be touching, and the highest person's opinion is getting weight over the others.
Adam
Yeah, absolutely.
Hugh
I mean, talking about saying no, try saying no to, to the higher ups about monetizing things.
Jess
Exactly. Because if you say no to monetizing, you're saying no to the success of the product overall, aren't you? Well, I mean, don't you want a job, I mean, this kind of... But that's, this is the thing. So some things need to stay local, you know, if we, if we argue for everything on a global scale, then it's really easy to find yourselves quickly going from discussing the use of a certain tech stack to whether or not we should be paid x pounds per month or what or whatever it's it's, we kind of need to keep things at a certain level to be able to argue for their categorical importance or unimportance.
Adam
So we've touched on this a little bit already. But let's talk a little bit about some of the methods and strategies that organisations can use to prevent scope creep from infiltrating their projects.
Jess
Sure, so scope creep, it's, it's a people problem, it's an organisational problem. So we need to look at how we're, how decisions are being made, and standardise that across the business. And I know that sounds like such an eye rolling exercise for everyone involved. But every single year, whenever the state of DevOps report is completed, it seems to find uniformly that when orders of interactions are defined, people become more satisfied with the outcome of their decisions. That's not to disrupt the org chart or the hierarchy of the people that you're working with. It's just so the expectations are set up so that people know how to have their voices heard. And people have a direct route to saying, to raising concerns about something, about about saying actually, I think this is going to, this is going to have us drift away from the main thing that we're focusing on. Another really important part is including feedback loops, and closing and making sure they're really fast. One of the merits of of working with production data is that you know what you're, you know that you're grounded in fact, whether we're talking from a sort of testing perspective, or, or just in terms of testing our, testing our theories, there is there is no production like production, we can't stage our decisions, and we can't stage our environments. So we need to keep things grounded in reality, and that means that we get tied to the same version of the truth. So testing in production is a great way to stop scope creep from happening because it prevents us from going down and into these imaginary realities where people will buy this feature that you haven't made yet, that doesn't necessarily exist and that might not even save the company keeping teams small, we've spoken about that. But the squad model I think is really interesting, because the squad model unites a small number of people like say eight people around one problem. And it forces you because because these teams are quite small and because the the the point of those teams is so specific, it causes you to create many more of those teams. And look at enablement teams way faster than you otherwise would. You stop having these big silos of teams and think, oh, gosh, we've now got this many engineers, we need to be able to support the engineers, let's have a supporting engineering team. And it's easy, it gets you away from having those layers, and it starts, it gets you to think about how to support the output a lot sooner. One, one thing that - it's really early on, but it's a project that I'm working with, with a couple of people - is around the idea of making it compulsory for team members to switch roles. A while ago, we saw that DoorDash was really, it was really insistent that everyone throughout their company, no matter whether or not they were on a super high pay packet or working in product, or if they had to have some sort of customer experience hours in in their contract. And I think grounding people in the actual work that you're, the actual kind of aim of the businesses is really important. But it's, it's about what form that takes, right? I mean, is it enough to just talk about your problems, is it enough to just to just have monthly meetings where the technical support team feed back the problems that that that that your users are having? Or do you actually need to be... how much do you need to consume your own cat food, dog foods, animal foods metaphor, for your activities to be productive?
Adam
I think that's a wonderful kind of idea. We were discussing a very similar thing with Ocado's James Donkin a couple of episodes ago about how when Ocado first started, basically, everyone did a couple of shifts in the warehouse doing the, you know, picking and packing and loading orders and everything. And I think that that is really vital, not just for, you know, customer service businesses, but for every, every organisation should have some mechanism for, as you say, grounding, you know, the workforce at large in what the organisation does, and what the priorities of of the organization's should be. When looking at, you know, expansion and, you know, new features and tightening up processes, you know, what are, what are the actual priorities, effectively?
Jess
I recently recently spoke to Patrick Debois, he works over at Snyk, and he's just a really interesting person, it was talking about how the majority of our problems come up when we tried to cram 100% utility into our teams, right? We try to make everything the peak of efficiency, and to meet all of our aims with all of our hours. And that's just not how we work, we need to break down - while we don't need to, we shouldn't be working in silos - we need to break down our time a little bit to make sure that we're sectioning off certain hours for being able to allow feedback to happen and to understand whether or not we're actually meeting the problem that we're trying to solve.
Hugh
Well, sadly, we're out of time for this week's episode. But thanks once again to LaunchDarkly's Jess Cregg for joining us.
Jess
Thanks so much for having me.
Hugh
Pleasure.
Adam
Always a pleasure to have you on the show, Jess.
Hugh
That was really interesting.
Adam
You can find links to everything we've spoken about today in the show notes and even more on our website, itpro.co.uk.
Hugh
Don't forget to subscribe to our social media and YouTube channel for more great content and leave the podcast a rating and a review.
Adam
We'll be back next week with more analysis from the world of IT. But until then, goodbye.
Hugh
