The Best Project Prioritization Methods and Techniques

written by Stuart Easton

Project prioritization methods: fake vs. realWhen you go to the doctor, you expect that she will use evidence and best practice when treating you. A doctor has a certain duty of care towards her patients.

It would be terrifying if doctors around the world just started making stuff up. I mean you’d run a mile from a doctor who said, “Got a flu? I reckon a few purple pills should do the job. I mean, they worked fine for Mrs Jones’ sciatica the other week!”

But that’s exactly what organizations (and PPM vendors) around the world do when running project prioritization. They make it up. They just put their thinking caps on and slap together a spreadsheet (or, even more scary, vendors slap together a module for their PPM tool).

Yet executives and managers have a duty to take care of every bit as binding as that of a doctor. They have a duty to use the organization’s resources efficiently in support of key organizational goals, but “making it up” has been a standard practice for years.

Luckily, we now have real evidence-based research to tell us how things SHOULD be done.

This is going to be a pretty ruthless, hard-hitting blog. If you read this, you will never be able to say, “I didn’t know…” again. You have been warned.

There is a "best way to prioritize projects" 

Academic research by the University of New South Wales1 into over 100 methods of running project prioritization came to a rather startling conclusion; only two of those methods are suitable.

And, unless you’re a regular reader of my blog, you’ve probably never heard of them.

These two processes are called AHP and DEA and if you're not using one of these methods, you're probably not getting good results (and you're certainly not following best practice). Let’s consider why many of you will be feeling rather uncomfortable with this finding.

The fact is that organizations have been picking portfolios for years. Some teams are more mature than others, but most are reasonably comfortable with their process. It feels good to make decisions the way you’ve always done just as it felt good to 18th century doctors to crack open a jar of leeches to treat a fever.

It’s what we’ve always done.

But it doesn’t really work.

Why do traditional methods fail? Why is AHP different? The answer comes down to how human beings make decisions and to how we make decisions in groups. Essentially, most methods of picking projects don't really build agreement and buy-in between stakeholders. That's why people by-pass the process and force in their own pet projects. That's why you end up with too many projects. 

What really puzzles many is that they believe that they are following good practice. They have a spreadsheet with weighted criteria. They've built a prioritization matrix. Surely that's what we need?

Nope. That's not what the research says. 

Your stakeholders may sit in the room and smile as the CEO dictates the weights of criteria, but they're not buying in. Because they're not bought in, they don't trust your scoring model and, essentially, ignore it. 

There's not really space in this blog to ago in to the detail of how methods like AHP solve this problem, but they do. AHP is based on decades of psychology research and decision science research and is designed to deliver both a good model (a "good quality decision") and a strong strong sense of ownership and buy-in to that model.

And that's why the research over the last couple of decades points to AHP (and DEA) as being the only two "suitable" ways to do project prioritization in a large organization.

This on-demand white-board webinar will give you more detail, including a high-level overview of the science, if you're interested.

Don’t get me wrong. If you work in a spreadsheet, or in a PPM tool,  you do get a portfolio, but a “made up” process is unlikely to get you an optimal portfolio (in any real sense of the word) and the cost of a sub-optimal portfolio is failing projects, missed opportunities - real impact on people's lives and on your organization's performance.

On the other hand, using one of the “suitable processes” (they’re called AHP or DEA, by the way) means that you have a framework that is designed to deliver a good decision with strong buy-in and support from the whole organization, one that allows you to optimize your portfolio and that is based on data.

Wrong ways to prioritize projects

So what are the ways in which people typically prioritize projects? What are these methods that are ineffective? This list sets out the most common:

Method Pros Cons
Allocate budgets to departments to spend how they like.
  • Allows each department to set its own priorities
  • Departmental priorities tend to be poorly aligned with each other
  • Mish-mash of projects, some pointing “in opposite directions”
  • Lack of process and oversight leads to inefficient use of resources
  • Subject to “local biases” (e.g. this just blew up – we have to fix it as top priority for our department to work”, but it might not be so important at the corporate level)
  • You haven't solved the business alignment problem, all you've done is given it to someone else who is solving it "the wrong way"
Exec prioritization meetings where an each project is "pitched" to the executives
  • Simple to understand
  • Executives like to flex their muscles
  • Very time-intensive for executives
  • Decisions often made for political reasons 
  • Projects get selected because they are "sold well" not because they are important
  • Typically, “loudest voice” gets projects – not driven by true business priorities
  • Open to pet-projects sponsored by powerful people
Just let the CEO or COO decide. They are the most powerful person so they should just make the decision.
  • Quick and clear
  • CEO or COO feels good!
  • Any single decision-maker is biased and influenced by their own experience - unlikely to really reflect needs of the organization
  • Mish-mash of projects, some pointing “in opposite directions”
  • Strong potential for pet projects to get through by lobbying the decision-maker
  • Lack of process and oversight leads to inefficient use of resources
  • Subject to “local biases” (e.g. this just blew up – we have to fix it as top priority for our department to work”, but it might not be so important at the corporate level)
  • Unlikely to have balance in the portfolio (across departments, business drivers, etc.)
Pick projects that deliver financial benefit
  • Everyone understands RoI or NPV!
  • Seems clear and quantifiable
  • It's not really clear and quantifiable. RoI, NPV etc are just models, estimates and are, generally, very unreliable
  • This method skews selection to short-term benefits with long-term consequences
  • Using pure financial metrics to select projects often undermines an organization's ability to achieve strategic goals.
Spread-sheet based project scoring and the project prioritization matrix
  • Cheap
  • Lack of mechanism to build consensus around criteria weights means this is really just a black-box with poor buy-in
  • Politics creep in again as the spreadsheet owner potentially gets to “manipulate” the outcome
  • Does not scale with the number of projects or number of “evaluators” of those projects
  • Often fails outright with pet projects getting approved and with too many projects being signed off
PPM Prioritization Module
  • Nice pictures
  • Integrated with PPM
  • Not based on any credible methodology
  • Nice charts make it look "scientific", but it's not - dangerous
  • Lack of mechanism to build consensus around criteria weights means this is really just a black-box with poor buy-in
  • Politics creep in again as the spreadsheet owner potentially gets to “manipulate” the outcome

If you are a Spreadsheet Jockey, then let's be clear - your spreadsheet is not as effective as doing it properly using value-based prioritization and AHP. And organizations that prioritize and actively manage the portfolio can achieve 40% more value than those that don't (McKinsey). 

Now you know, it’s time to act

How would you feel if your doctor prescribed your child a drug to treat a potentially fatal disease – a drug that has been on the market for years but that is simply less effective than a new drug that was just released – just because it’s what she’d always done?

You wouldn’t stand for it, would you? I know I wouldn't.

So, now you know that "your drug" doesn't really work and there is a "right way" to prioritize and select projects. Research has shown AHP and DEA to be the most suitable methods for project prioritization.  Can you, in all conscience, ignore this new-found knowledge?

So, where should you start?

I can point you at resources to learn about the analytic hierarchy process (to give AHP its full name). I’m such a big fan, in fact, that we built our project prioritization product around AHP.

I can point you at resources that describe how project success rates are strongly influenced by the project prioritization process.

You can even download an infographic showing the most common signs that your prioritization process is broken.

But only you can decide to engage with those resources and then to act.

But you can no longer pretend that you don’t know. And in medical circles, failing to treat patients properly when there exists “established best practice” has a name. It’s called malpractice.

Don't be guilty of PMO malpractice.

 

1. Danesh*, Ryan and Abbasi (2017), School of Engineering and Information Technology, University of New South Wales