Decision making on the iPhone

Sometimes it's hard to make decisions. In my experience, a hard decision is one that involves multiple criteria the relative importance of which aren't particularly clear. There exist methods to help you make such decisions. One of them has been implemented in an iPhone app called iDecide+. So I thought I would write about decision making, and briefly review the app, at the same time.

Consider a simple example: taking a vacation. You would probably want to spend as little money while having as much fun (however you define "fun") as possible. But there's more to it. You might want to go somewhere you've never been. You might prefer to distinguish between the costs of travel versus the costs of activities (beer, sightseeing, souvenirs, nightclubs, whatever). You might want to go somewhere with a particular climate. These are some of the criteria that would help you decide which vacation is probably the best for you.  And these are just some of the possibilities; there are many others.

It gets worse: some of the criteria are more important than others. For instance, the cost of travel might be more important to you than the climate at the destination. How do you juggle mentally all the criteria and how important each is, and still be able to evaluate all the alternatives consistently?

The brain is really bad at juggling so many variables at once, so it's quite likely that you would have a very hard time choosing a vacation with confidence.

Making decisions

The standard method for dealing with these problems is divide and conquer: separate the problem into a bunch if smaller problems, each one of which you can solve relatively easily because you can focus on only a couple of variables at a time. The real trick, though, is to break the problem apart so that the solutions fit back together as easily as possible. Otherwise, what you gain by dividing and conquering you will lose in having to reassemble the results.  While there are lots of ways to break problems down, but it's very hard to find a breakdown that will let the constituent solutions fit together easily at the end.

One method that I really find useful is the analytic hierarchy process, or AHP. AHP comes in all kinds of flavours and dialects, but they're all based on six basic elements.

Establish criteria. The criteria are desirable characteristics that will be exhibited to the greatest degree by the best alternatives. They will be the things about each alternative that you will measure. Ideally, the criteria should be completely uncoupled. Uncoupled criteria are far easier to work with. For instance, in our vacation example above, we separated travel cost from the cost of everything else; these costs are uncoupled. If on the other hand we had set as criteria (1) travel cost and (2) total cost, then the two cost criteria would have been coupled. Unfortunately, depending on the situation, you may not be able to find uncoupled criteria. In such cases, you just do the best you can.

Assign weights to the criteria. Since some criteria may be more important than others, you need to include that relative importance into your decision making process. To do this, you can assign a weight to each criterion. The larger the weight, the more important the criterion. There are some extremely reliable ways to assign weights; my favourite is pairwise comparison. It's extremely well known and described elsewhere, so I won't waste space repeating it here.

Establish a rating scale. The rating scale is the set of values from which you choose when you evaluate alternatives. Since the evaluation is not strictly a scientific exercise, there will be a significant vagueness to it. This isn't a problem so long as you don't assume more precision than you can get. One way to do this is to choose a fairly coarse rating scale. My favourite is a linear scale from -2 to +2. On this scale, 0 stands for a neutral rating, negative values stand for increasingly poor ratings, and positive values stand for increasingly good ratings.  I've always found this five point scale to be the easiest to use.

Identify a reference item. The reference item is your baseline item. It sets the value of "zero" for any evaluation. Setting a good reference is really important because to help control the uncertainty inherent in all decision making, it's good to have one fixed point from which to measure things. The reference item should be something well known, preferably "the way things are now" or have been in the past. If you're deciding on a vacation, a good reference is your last vacation. If you want to buy a new car, a good reference is the car you have now.

Identify the alternatives. Here's where you get creative. I'm not going to get into all the ways you can come up with alternatives, but this is where you do it. The important thing is that all the alternatives must be fleshed out to the same degree, or evaluating them won't be possible. For our vacation example, you'd need to know enough about each possible vacation to have the confidence to evaluate each one against each of the criteria uniformly. This doesn't mean you need lots of detail; it does mean that you have to know as much - or as little - about each alternative as you know about every other alternative.

Rate the alternatives. Now you look at each alternative in turn. For each criterion, you compare the alternative to the reference item, and score the alternative using the rating scale. So, for example, if a given alternative for your next vacation has a somewhat higher travel cost (and is therefore slightly worse) than your last vacation, you might rate it as -1 on that criterion.

Once you've rated a given alternative against all the criteria with respect to the reference item, you multiply each rating by it's corresponding weight. This gives you a weighted score for that alternative with respect to each criterion. Finally, you sum up all the weighted scores; this gives you a single score for the alternative.

Once you've done this for all the alternatives, the highest scoring alternative is your winner. And your decision is made.

Now, this may seem awfully complicated, but it's actually easier on your brain, and much more robust, than just trying to juggle mentally all the variables at once. It separates the overall task into two types: things the brain is good at doing, and everything else. The "everything else" part is packaged up into the method so that your brain is left to focus in the smaller, but most important, tasks.

An iPhone app to help decision making

Here's where software can help. If the "everything else" part is bundled up in software, then the user is really free to focus on the true cognitive tasks. The software would guide you through the method, prompting you only for the bits that need the human touch. Notice that with the right software, the overall cognitive burden becomes even lighter because all the administrivia needed to manage the AHP process itself is handled by the software.

iDecide+ is this kind of app: it walks you through the AHP process, letting you focus on the cognitive parts of the decision making process. It follows a process virtually identical to the one I outlined above.

As far as functionality goes, iDecide+ is perfectly fine. Computationally, there is no great rocket science behind AHP. It does exactly what it's supposed to, and it does it correctly. It prompts you for all the data needed, and gives you a very nice summary of things at any point in the process. You don't have to go all the way through the method to check the summary, so it's a bit like a running total of what you've done so far. And once you've completed a run, you can always go back and tweak your data; the effect of such changes is immediate.

The design of the app is clean and simple, which is just how I like my apps. However, the user interface has some shortcomings that can confound the first-time user.

Problem #1. It's sometimes hard to tell if you've successfully added entries to some screens (e.g. when you're adding new criteria) because the new item is added to the bottom of the list. If your list is already long enough, the new item will be off-screen and you won't know the item was added.

Problem #2. Another problem has to do with the weighting of your decision criteria. iDecide+ walks you through pairwise comparison. This means you will be asked to rate the importance each criterion with respect to each other criterion.

For example, if you have five criteria, you will have to do 10 comparisons. If there are six criteria, you'll have to do 15 comparisons. iDecide+ will walk you through all the comparisons, and then loop back to the first one (presumably so you can re-check and tweak the values).

The problem is that you can't tell that you've looped around and are redoing comparisons you've already done. You can end up going round in circles. Not very productive.

Problem #3. While you're rating one alternative, you cannot see how you rated previous alternatives. If you have a very crisp way to rate an alternative with respect to the alternative, then you're fine. But for vague decisions, it can be hard to be consistent. Yes, you rate an alternative with respect to the reference item, but that isn't necessarily enough. Being able to see all your ratings for all your alternatives can be very useful.

But iDecide+ doesn't do that. It only shows you one alternative at a time. I've found that I can rate the alternatives faster if I write down the ratings on a single page and then transcribe them into iDecide+.  It helps me be more consistent in my rating of different alternatives.  It would be good if the user interface let you see all your ratings at once. Granted this is hard to do on the relatively small screen of the iPhone, but I'm sure there's a way.

Problem #4. It's hard to change the rating scale. There are two scales built into the app: a 10-point linear scale from 0 to 9, and an exponential scale from 0 to 9 (that is: 0, 1, 3, and 9). Since iDecide+ let's you set a different scale for each criterion, you need to set them all manually. This can be quite onerous, especially if you're like me and have a preferred scale for everything that isn't one of the builtin scales (like my -2 to +2 scale).

In the end, though, iDecide+ is still quite a useful tool because AHP was carefully thought out.  iDecide+ takes a lot of the gruntwork out of making better decisions.  If used diligently, AHP can be quite beneficial and if its user interface were to be improved to address the problems I've noted, I think iDecide+ would become quite excellent.