How to Deal with Inconsistent Comparisons in Analytic Hierarchy Process

Written by Dawid Opydo


“I am depressed. I entered my pairwise comparisons but the AHP software claims that they are inconsistent. I have no idea what this means. I was trying to change some comparisons but I think I messed it up even more. Please help me. I know Analytic Hierarchy Process is a powerful tool, but right now, I’m in a real mess!”

If you have ever felt this way, this article is for you. You will find here tips on how to deal with inconsistent comparisons. Don’t worry, it’s not difficult.

Ok, first things first…

What is this “inconsistency”?

 

Imagine that the software asks you to enter 3 comparisons. You answer that:

A = 3*B

B = 2*C

You will be totally consistent only if you answer that A = 6*C. Any other value will be inconsistent (a little, if you say that A = 5*C but more if you say that A = C).

In our simple example, you only had three things you were comparing. In most cases, you will have more than three comparisons and there is bound to be some degree of inconsistency.

The TransparentChoice AHP software will warn you that inconsistency is higher than acceptable – the circles below your judgments will go orange or red. If you are still reading this article then probably you’ve already seen this…

What causes inconsistency?

Mistakes. A simple lack of concentration, or pressing a wrong button can introduce inconsistency. We are human and it is easy to make mistakes when you enter several comparisons.

Lack of knowledge. When we don’t have enough information to make consistent comparisons, or we are uncertain. In this case we use our judgment, and our judgment is sometimes not as accurate as we’d like.

Human nature. We are human beings, we are inconsistent. This isn’t a problem, so long as we’re not too inconsistent.

Discrete scale. In AHP, we do pairwise comparisons using a special scale which contains the values 1 to 9. This can introduce inconsistency. Imagine that you answer that A=2*B and A=3*C. The totally consistent value for comparison B vs C is B = 1.5*C, but since 1.5 is not on the scale, you must be inconsistent. This will only have a marginal effect on the total consistency, especially if the number of comparisons is not very large. Sometimes you simply don’t have a chance to be totally consistent…

Capped scale. Imagine that you answer that A=4*B and B=5*C. To be consistent you should answer that A=20*C but maximal value that you can use is 9. Again, the AHP methodology forces you to be inconsistent.

How to resolve problems with consistency?

Avoid more than 9 elements to compare. It is easier to be consistent on smaller set of comparisons, and the masses of research into AHP suggests that you should try not to compare more than 9 items using pairwise comparisons. This blog can help you.

Proper hierarchy. Don’t compare elements that are extremely different in priority/weight. Remember that 9 is maximum on the scale and it should be enough. If one item is more than 9x as important as another, they probably don’t belong on the same level of your hierarchy.

Don’t overuse large values. If you overuse large scores – you will have problems with capped scale. Remember, if you select a score of 9, you are saying that A is 9x as important as B.

Eliminate contradictions. There are some obvious contradictions that you might make. For example, if you say that A > B > C > A… well, there’s a contradiction that needs to be sorted out. TransparentChoice helps by alerting you (red circle below the contradictory judgment). Simply go back and look at your judgment to resolve the inconsistency.

Inconsistent comparisons in Analytic Hierarchy Process

Reduce the inconsistency of your comparisons. TransparentChoice, and the AHP methodology it is based on, has a mechanism to measure consistency. It is called consistency ratio (CR) and as long as this is < 10%, you don’t usually have to worry.

Of course, doesn’t mean that 11% is always unacceptable. If CR > 10% software will help by giving you a warning and it will indicate comparisons that seem most inconsistent. If you review those judgments you will often find that the score is not really what it should be.

If your consistency ratio is still higher than 10%, the next two most inconsistent judgments will be flagged up for you to look at. Simply repeat this process until you’re happy with your judgments, or until you get no more orange flags.

Sign up for AHP Software

New call-to-action

Don't forget to share this post!