We all knew it
An Agile Project eh. Like an Agile Waterfall process? cool. Cool cool cool.
I know PMI has an Agile thing but by and large Agile can’t be “projects” and vice versa.
Cries in waterfall.
One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed.
I’d like to work in that company.
No thanks. It’s way more fun to be part of the decision process. If a manager can anticipate all of the requirements and quirks of the project before it even starts, it’s probably going to be a really boring, vanilla project at which point it’s probably just better to but the software.ä somewhere else.
Creating something new is an art in itself. Why would you not want to be a part of that?
Also: Isn’t it cheating to compare the two approaches when one of them is defined as having all the planning “outside” of the project scope? I would bet that the statistics in this report disregard ll those projects that died in the planning phase, leaving only the almost completed, easy project to succeed at a high rate.
It would be interesting to also compare the time/resources spent before each project died. My hunch is that for failed agile project, less total investment has been made before killing it off, as compared to front loading all of that project planning before the decision is made not to continue.
Complementary to this, I also think that Agile can have a tendency to keep alive projects that should have failed on the planning stage. “We do things not because they are easy, but we thought they would be easy”. Underestimating happens for all project but for Agile, there should be a higher tendency to keep going because “we’re almost done”, forever.
Try medical software and devices. The requirements and specs are mandatory before doing anything. It’s actually very fun and I have less burnout thanks to this.
I couldn’t disagree more.
In medical I would end up being apart of endless retirement gathering meetings, then draft up the SOW doc only to have stakeholders change requirements when they were reviewing the doc. Then months later once the doc was finally finished and I could do the development, when UAT time finally came, they’d say the build wasn’t what they wanted (though it matched the written requirements).
Most of the projects I saw executed in the last 4 years either got scrapped altogether or got bogged down in political bs for months trying to get the requirements “just right”.
It was a nightmare. You could blame me, or the company, or bad processes all you want, but I’ve never had fun on a waterfall project, especially not in medical. (Though, in my opinion, we are severely understaffed and need like 4 more BAs.)
It’s almost like the methodology is less important than the people.
Do you think the problem is that the person driving the requirements doesn’t know what they actually want?
I think a good BA is critical to the process because lots of end users have no idea how to put their ideas onto paper.
I also think an MVP helps a lot because people can see and touch it which helps focus their needs.
I would say yes, the problem is stakeholders not having thought critically about what they really wanted from the project.
The motivation for projects were usually “regulatory told us we need to have this new metric for federal reporting”, or “so-and-so’s company can do this, why can’t ours” rather than, “we’d like to increase retention by 6% and here’s the approach we’ve researched to make that happen”.
I ended up experiencing that people in the highest positions weren’t experts in their field, but just people who had a strong intuition. This meant they would zero-in on what they wanted by trial and error rather than logic. Likewise, it meant they were socially adept enough so their higher-ups would never get mad at them when we finished “late and over budget”. People lower on the totem received that blame.
I think humans are just really bad at estimating and keeping their commitments, which is why I enjoy working with agile more. It’s a forgiving framework (imo).
That’s because they forgot the meaning of the word agility and want to apply the rules what ever the cost
A lot of places seem to view it as “we just work from the backlog” with no requirements on when features are delivered, or their impacts on other parts of the project.
You still need a plan, goals and a timeline. Not just a bucket of stuff to get done.
Or, even worse, they want to apply some of the rules, cherry-picking bits and pieces of a framework without truly understanding it.
And also because it’s a comfortable cover up for any kind of money saving stupidity. We don’t need proper requirements engineering, we’re agile. We don’t need an operations team we’re doing an agile DevOps approach. We don’t need frontend Devs, we’re an agile team you all need to be full stack. I have often seen agility as an excuse to push more works towards the devs who aren’t trained to do any of those tasks.
Also common problem is that still tons of people believe agile means unplanned. This definitely also contributes to projects failing that are just agile by name.
100% my experience.
Especially the last part. Writing a single word into a jira ticket doesn’t make it a story, epic or sub task. You’re too lazy to specify, that’s not what agile is meant to be.
I don’t know how many times I have been waiting for the product manager to get out of their meeting so they can help me clarify what they really mean with their “high priority” ticket only consisting a vague title.
“Looks like I’m home early, bye”
This is the best summary I could come up with:
Even though the research commissioned by consultancy Engprax could be seen as a thinly veiled plug for Impact Engineering methodology, it feeds into the suspicion that the Agile Manifesto might not be all it’s cracked up to be.
One standout statistic was that projects with clear requirements documented before development started were 97 percent more likely to succeed.
“Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout.”
A neverending stream of patches indicates that quality might not be what it once was, and code turning up in an unfinished or ill-considered state have all been attributed to Agile practices.
One Agile developer criticized the daily stand-up element, describing it to The Register as “a feast of regurgitation.”
In highlighting the need to understand the requirements before development begins, the research charts a path between Agile purists and Waterfall advocates.
The original article contains 502 words, the summary contains 175 words. Saved 65%. I’m a bot and I’m open source!
what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout.
I haven’t read the book they’re advertising here, but I’ve found these challenges to be socially created, not caused by agile.
Oh, so Agile is only done by autonomous AI?
In the article, they’re proposing a solution to agile (“Impact Development” or something). The quote I listed above is talking about how Impact Development is supposed to provide those things. That said, I don’t blame agile for projects not having those things, it’s the people’s fault. So changing methodologies likely won’t help.
In short: yes, make AI do all project management :P
Does that surprise me? Not at all. “Agile” was never about making programming better. It was a management buzzword from the start.
We once had a manager who came to me with the serious idea “to make the development process agile”. He had heard of this in a discussion with managers from other companies. The problem? I’m the only person in this department. I program everything alone. How the F should I turn my processes “agile”?
Work for two weeks, take two weeks off to think for a bit, work for two weeks. Rinse and repeat. That’s what agile is, right?
I think he wanted it more like Product Owner, Scrum Master, Architect, Stakeholder, New product development, Tester, Integrator, Team member, Agile architect, Agile Coach, Developer, Team lead, Technical expert, Product Designer, Business Analyst, Programmer, and Specialist for at least eight hours a day in each role…
If you know exactly what you need, then specs are great. Proven solutions for known problems are awesome. Agile is pointless in that circumstance.
But I can count on one hand the number of times stakeholders, or clients, actually know what they want ahead of time and accept what was built to spec with no amends.
When there is any uncertainty, changing a spec under waterfall is significantly worse. Contract negotion in fixed price is a fucking nightmare of the client insisting the sky is red.
If you know exactly what you need, then specs are great.
If you know exactly what you need and the specs are great, then you barely need project management framework at all.
Maybe I just work at shit companies, but it feels unrealistic to expect this this level of maturity from assigned work.
Well, exactly.
Feels like the old php metric. PHP had a ton of great code and successful projects but it also attracted very bad devs as well as very inexperienced devs leading to a real quality problem.
Honestly kinda see thing in a lot of JavaScript applications these days. Brilliant code but also a ton of bad code to the point I get nervous opening a new project.
My point? It may be a tough pill but it’s not the project framework that makes projects fail, it’s how the project is run.
Agile falls into the category of how the project is run
No it’s a set of tools you can use to run a project.
My point is that a lot of people use “agile” to mean not planning or don’t put guard rails on scope and they fail. That’s not agile, it’s just bad PM
Agreed.
Being Agile is being flexible. To do that you need to plan for multiple contingencies. Resulting in more planning, not none.
“agile” is being flexible. Being “Agile” more often than not means your company’s incompetent management paid some hack consultants to come in and bless your flavor of stupid bureaucracy as “Agile”.
I witnessed a huge number of failed projects in my 25-year career. The cause was almost always the same: inexperienced developers trying to create a reusable product that could be applied to imagined future scenarios, leading to a vastly overcomplicated mess that couldn’t even satisfy the needs of the original client. Made no difference what the language or framework was or what development methodology was utilized.
I feel like that’s the same underlying issue: The requirements are not understood upfront.
If a customer cannot give you any specific information, you cannot cut any corners. You’re pretty much forced to build a general framework, so that as the requirements become clearer, you’re still equipped to handle them.
I guess, the alternative is building a prototype, which you’re allowed to throw away afterwards. I’ve never been able to do that, because our management does not understand that concept.
I feel like that’s the same underlying issue: The requirements are not understood upfront.
Actually on most of these failed projects the requirements of the original customer were pretty clear. But the developers tried to go far beyond those original requirements. It is fair to say that the future requirements were not well understood.
the alternative is building a prototype, which you’re allowed to throw away afterwards
Lol I’ve done many prototypes. The problem is that management sees them and says “oh, so we’re finished with the project already? Yay!”
I’ve seen a lot of contractors over promising timelines too. “No matter how hard you push and no matter what the priority, you can’t increase the speed of light.”
But yeah exactly.
Yeah, look at the most prolific language at a given time. There’s your crappy projects or your soon-to-be-crappy projects. What are the universities and ‘coding academies teaching’? That’s going to be the crappiest stuff in the world when those students come out.
So too it goes with ‘management’, the popular ‘self-help’ style crap of the moment is what crappy teams will adopt, and no matter what methodology it is, that crap team is still crap, and it will reflect on that methodology.
The few times I’ve been on an agile project it amounted to start writing without understanding what product we’re building.
Yeah. Which actually doesn’t have to be bad as long as leadership accepts that this exploratory work (sometimes called a “spike”) might have to be thrown away, if findings reveal better paths.
The trouble begins when you start shipping your proof-of-concepts (without immediately paying back that tech debt).
It very quickly becomes an unmaintainable mess.
I think I would also need to see the point at which agile projects are scrapped vs waterfall and how much money is sunk into them by time of scrapping.
My company knows agile will fail more often but also that they fail earlier. So they take on more projects and those seemed to be a bit riskier compared to what they would take on if it were to go by waterfall process.
I am not an agile acolyte, but failure % alone is not convincing. “Fail early, fail often” is a common mantra for a reason.
I haven’t read the article yet, but surely they can’t be juxtaposing waterfall as the alternative to agile. The modern alternative, especially in small to medium businesses, would be kanban.
Kanban is Agile. They are pushing Impact Engineering.
Well that’s news to me
Ehhhh…Kanban is much older than Agile even if they tried to subsume it and say it’s an agile technique, so that’s sort of right. But kanban vs “scrum” - which virtually everyone means when they say “agile” - is fair.
Within my company there is a mix of Scrum and Kanban, so Agile != Scrum.
I don’t think it makes much sense to say “We are switching from Agile to Kanban”, but “We are switching from Scrum to Kanban” does make sense (at least to me)
A more proper title would be “study finds 268% higher failure rates for poorly planned software projects”.
“Agile” is mostly an excuse of poor planners for their poor planning skills.
I don’t have much direct experience working in agile since I tend to work on the business side but I can tell you that the term agile is WAY overused. So many projects are described as agile when they are just waterfall with more steps. Leaders love to say they are working in agile because it sounds ‘techy’ and cool, but I don’t think they fully appreciate what it is vs other methods. I wonder if a lot of the failed projects described in the article are some of those agile in name only kind of things.
Agreed. The problem is people mistake “zero planning and structure” to mean “agile”. Of course it fails.
Agile to me was always mini waterfall. You always know who’s doing what, why, and what success looks like on a 2 week sprint horizon. When you see people on a sprint without a clear understanding of what they are doing over the next couple of weeks - then you know your project is in trouble for sure.
An even better title would be “‘Study’ by firm pushing new technique finds old technique is bad.”
Yeah, Agile isn’t really at fault here. If done right - if you’ve got a scrum master, a proper product owner, proper planning and backlog grooming, etc. - it works really well. The problem is some companies think Agile is just “give the devs some pie-in-the-sky hopes and dreams, let 'em loose, and if they don’t give half a dozen execs exactly what they want (despite their massively conflicting ideas on what they want), cancel the project.”
Yeah, Agile isn’t really at fault here. If done right
This is what ticks me off about the “Agile” brand, it’s chock full of no true Scotsman fallacy (if a team failed while doing “Agile”, it means they weren’t being “Agile”).
I can appreciate sympathizing with some tenets as Agile might be presented, but the popularity and consultancy around it has pretty much ruined Agile as a brand.
Broadly speaking, any attempt to capture nuance of “best practices” into a brand word/phrase will be ruined the second it becomes “popular”.
This isn’t a case of No True Scotsman. There really is a right way and a whole lot of wrong ways to do Agile development. Any team that calls itself an Agile team that doesn’t actually follow the processes properly is doing it wrong and will fail.
That doesn’t mean any team that’s doing it right will succeed, but it’s like riding a horse: If you only climb halfway up the horse and try to hold on while at a 90-degree angle, it’s not going to work, and it would be stupid to declare that the concept of horse-riding is broken. No, it’s not broken, you’re just an idiot who thought you could ride a horse while only halfway up, clinging desperately to its side.
In my experience it’s just kanban, but make the devs feels guilty between sprints for not meeting their goals.
Absolutely It’s so management can say “your velocity was down 15% this sprint” and not feel bad about it instead of saying “work more” It’s plausible deniability for demanding unpaid overtime
In one the worst “poor planning” projects I’ve been in the product owner just kept sneaking in new “high priority” issues to the top of the backlog throughout the sprint. I don’t think we had a single sprint where we ended up with fewer open issues in the backlog than when we started.
Needless to say, he was the main reason why I quit.
As someone who practices agile software development I find this baffling. I’ve never started a new project without at least 3 weeks worth of research and requirements gathering (and obviously more as the project rolls on). There are seriously companies out there who are like, “Mmm, I feel like this is gonna be an Electron project. Let’s just lay the groundwork and we’ll flesh out the nitty gritty in a week or so.” 😱
Wtf is Agile ? I can’t get my head around that.
It is a methodology to develop software quickly. It has some good things about it. But it can be very heavy on meetings and agile idealists are not very flexible. As many of the other comments say, a mixture of agile and some other methodology or starting with agile and developing your own process that works for your team or project is the best way of managing a project. I don’t understand why so many people don’t seem to write requirements when using agile. Even with agile I will not start coding until I have relatively clear requirements. It is not too bright to start speculative development without really knowing where you are going. https://agilemanifesto.org/
But it can be very heavy on meetings and agile idealists are not very flexible.
Seems a little ironic haha
Right? I find agile purists to be some of the least flexible people I’ve ever met. They are the exact opposite of agile. To be fair though, I have found that a good scrum master can be worth their weight in gold. You always know the status of a project and the individual stories. It can be very, very helpful.
I don’t understand this… How do you code if you don’t know where you’re coding for? Am I the only one that thinks that sounds crazy?
Commonly you will have a relatively broad goal of providing some functionality by the time a project is done. Every sprint, commonly two weeks, you concentrate on producing a piece of functionality that will get you closer to that goal. At the end of a sprint, many teams are expected to have what’s called a minimally viable product that is technically usable. The problem with that concept is MVP almost always becomes production. That results in poor coding that is hard to support. It almost always involves rework later on, often when something is already in production. And you are not crazy. Not having a clear idea of what you’re coding for is wasteful and very inefficient.
Agile is not about being quick, it’s about delivering what the customer actually wants. When you do Waterfall you gather all the requirements, then you sit down and code the thing. Only to find out months or years later that you delivered crap as the customer didn’t even know themselves what they wanted.
With agile you take it one step at a time. What is important now? Get the requirements for this feature, deliver it in the next two weeks (or at least a part of it). Then the customer, which can be an actual customer, or your internal Product Owner, or a Product Manager looks it over. If the whole thing is perfect? Nice, carry on to the next thing.
Often you find out some detail was overlooked, or a new requirement came up, or the design didn’t fully work out. So pack it into the next sprint and do it better. You’d never get this feedback if you gather “all” requirements first and then just try to go from start to finish.
Agile certainly has its upsides when done right, unfortunately there’s not a lot of companies who manage to do so (like most I’ve been part of). Despite being messy at times, it’s still better than Waterfall. There’s too many meetings either way.
Pbpbpbp…agile fails fast by design.
The counter from the article is you need a specification first, and if you reveal the system wasn’t going to work during requirements gathering and architecture, then it didn’t count as a failure.
However, in my experience, architects are vastly over priced resources and specifications cost you almost as much as the rest of the project due to it.
TLDR…it’s a shit article that confuses fail fast with failure.
Fail fast is the whole point and the beauty of agile. Better to meet with clients early and understand if a project is even workable rather than dedicating a bunch of resources to it up front and then finding out six months in (once the sunk cost fallacy has become too powerful)
Thanks for pointing that out so I didn’t have to.
What’s the alternative? Waterfail?
Yeah because business requirements and technology is changing at an ever slower rate…
