Projects Fail - Blame the PM?

Over 50% of projects fail.

Of course, if we had good project managers, we’d never have failed projects, right?

Not right.

We now know that PMs should not carry all the blame for failed projects.

Research suggests that many projects are doomed to failure long before project managers even get involved. Around 40% of the “cause” of failed projects occurs before projects are even initiated.

Since a PM cannot normally travel back in time and get involved in a project in which they are not yet involved (there’s a sentence for Terry Pratchett fans!), I’d suggest someone else should be driving that particular pre-execution bus.

And that someone should be the PMO (or EPMO).

How many projects fail?

This is a tough one to answer. It sometimes feels like every vendor is releasing yet more survey data showing just how bad project failure rates are, but vendors have an agenda: like insurance companies, vendors want to scare you into spending money.

Academics, on the other hand, have no such agenda and ought, therefore, to provide an unbiased assessment of what project failure rates really are. So I set off to find some real research data and it turns out that there is no really reliable data on project failures. Different authors define failure differently. Some industries and sectors feel that a 20% overrun is perfectly fine, for example, while others set the threshold at 10%.

In one paper, published in 2015, the researcher reviewed multiple studies into project failure rates and concluded that data appears to indicate an overall failure rate of between 50% and 70%, but he emphasises that the data is not great.

Let’s take the mid-point between 50% and 70% - assume 60% of projects fail - and see where this fits into the wider universe. Put another way, 60% failure is the same as projects succeeding an average of only 40% of the time.

According to PMI data, the best organizations successfully deliver their business benefit 92% of the time. That’s quite a difference.

These same superstars deliver projects on budget 90% of the time and they deliver projects to schedule 88% of the time.

Having said that, the 40% success rate of the average project team looks good compared to the worst organizations out there. You can see all the data in this infographic.

So, having high failure rates is not inevitable. What is it that the superstars do differently?

40% of failures are not about poor execution

Of course, those top performers are executing their projects well, but that’s not enough. According to academic research pre-execution causes of failure comprised nearly 40% of the total.

Let’s say that again, this time with feeling…

40% of the causes of project failure happen before the project even begins!

In other words, you can execute your projects faultlessly (and, let’s face it, this is where most PMOs spend 95% of their time), and still fail.

Fail a lot.

So, does this mean that investing in things like PM training, PPM tools, templates, etc. is pointless? No, of course not. Anything you do to improve delivery is a good thing, but it can only take you so far.

Okay, let’s say THAT again, in different words, and this time with feeling…

You CANNOT execute your way to high project success rates!

Look at it this way. The rule of diminishing returns applies to project execution just as is does to pretty much everything else. You can invest, say, $500k and get 80% the way to perfect execution but getting to 90% might take another $2m. But 40% of failure has nothing to do with execution so that $2m is not a great investment.

So rather than chasing those last few percent on execution, let’s look at what else we can do to increase project success rates.

How projects fail before they start

I've talked a lot about how strategic alignment is vital if you want to improve project success rates. This has mostly been based on data from the PMI (which you can access via this link). This empirical data that shows a very strong correlation between alignment with strategy and project success.

Now it turns out that researchers have come to the same conclusion by looking at the problem the other way round. They looked at the causes of project failure.

As mentioned before, there are no great data on the percentage of failed projects. It's pretty easy, however, to look at failed projects and to figure out what contributed to that failure... and that’s exactly what researchers have done. One of the leading researchers in this field is Alan Stratton and he just published a paper pulling together data on the causes of project failure together with a framework to help identify where in the organization you should look to fix these failings.

And this is where that 40% of failure being caused pre-execution comes from. Weaknesses in the strategic planning process - defining strategy, coming up with project ideas, evaluating ideas, prioritizing projects, etc. - is the root cause of project failure. Get the whole strategic alignment process right and you can eliminate much of this risk.

Of course, there are the “normal” execution-related project failure modes, but the ones that interest me in this blog are the ones that occur before the project has even begun (green in the image below).

There’s a lot of detail in this model - it’s a bit of an eye-test, in fact - but the take-away messages are;

• Many organizations fail to clearly define their strategic goals and drivers in a way that can be used to prioritize and select projects. This lack of project prioritization often means that projects make it through that really don’t add value. This reduces the focus on truly important projects and can even steal their resources.

• Assuming there is an overall strategic direction and that it’s used to select projects, that information frequently does not make it through to the project team. This means that the team cannot focus on delivering benefits vs. delivering features - another major failure mode. It also means that it's harder for the team to make good in-flight decisions that support strategy.

• Project ideation (aka sourcing projects) is broken in most organizations. Ideas should start with “what are our strategic goals”. This should trigger a process of brainstorming, evaluation and improvement of the project concepts that support strategy. Most organizations simply respond to “demand from the organization”. Don't get me wrong.  You have to listen to the business, but the PMO is in the unique position of being able to help the business help themselves. You're an impartial participant and you are one step removed from the day-to-day pressures that can bias managers towards tactical action.

• Estimation is broken in many organizations. This is often about a lack of critical questioning and a lack of exploration of potential “landmines” within projects. The recommendation from the research? Get a senior PM involved in evaluating and estimating projects.

Okay, time for the eye-test. The green parts are the pre-execution factors that influence project failure rates.

Whose problem is this to fix?

The researchers recommend that “project management” should own the fix for all of this. They go on to say that, sadly, many project management organization leaders - you and I might call them PMOs - lack the skill to do so.

The researchers are not suggesting that PMOs take ownership of the strategy. What they're saying is for the PMO can own the process and tools that will help the business make better decisions.

Don’t get me wrong - I’m not suggesting tools are are a magic bullet that will fix everything, but they can be effective scaffolding to massively improve various parts of the process and to provide the structure that can allow a PMO leader to succeed.

So, where to start?

Here are some recommendations to get the ball rolling;

1. Quantify the problem: Doing a quick portfolio review will help you clearly articulate to senior stakeholders why you need to change the whole strategic formulation and strategic development process (demand management, project evaluation and project prioritization in project speak). Your senior leadership are the ones who “own” strategy, but the PMO should own the “process”.

2. Win exec sponsorship: Now the problem is visible, it’s time to paint a picture of how to “fix it”. TransparentChoice has a handy “How to Prioritize Projects” guide that might help you frame things up.

3. Win PM support: It’s time to ‘fess up to the PMs. Help them understand that they have a responsibility to execute well, but that the organizational leadership has a responsibility to define strategic direction, to ensure projects support that direction and that projects are adequately resourced…. But the quid pro quo is that they, the delivery teams, then focus their work on delivering business benefits rather than checking off functionality.

Does this work? Yes!

Coming back to that PMI research we learn that the biggest difference between “normal” project delivery organizations and the super-stars, the ones that deliver results 92% of the time, is that these top-performers have PMOs (or EPMOs) who focus on strategic management of the portfolio… just like the academic research framework predicts.

So coming back to the title of this blog, are PMs to blame for project failure? Well, yes, partly, but the PMO and the organization’s strategic leadership have to raise their game as well. Without that changed your project success rates can never even approach the top performers no matter how good your execution is.

The PMs can’t do it on their own.