Basic Concept of Probability

The probability of an outcome is the proportion of times the outcome would occur if we repeated the procedure many times. Chance that an event will occur. Theoretically for equally likely events, it is the number of ways an event can occur divided by number of outcomes in the sample space. Empirically, the long term relative frequency.

 Examples

 Coin: What is the probability of obtaining heads when flipping a coin?

 A single die: What is the probability I will roll a four?

 Two dice: What is the probability I will roll a four?

 A jar of 30 red and 40 green jelly beans: What is the probability I will randomly select a red jelly bean?

 Computer: In the past 20 times I used my computer, it crashed 4 times and didn’t crash 16 times. What is the probability my computer will crash next time I use it?

Some useful terms:

 Experiment - Any happening whose result is uncertain.

 Outcomes - Possible results from an experiment

 Sample Space - Set of all possible outcomes a process

 Event - Subset of the sample space. One or more outcomes.

 Equally Likely Events - Events which have the same chance of occurring

 Independent Events - Events in which the occurrence of one event does not change the probability of the occurrence of the other. One does not affect the other.

 Dependent Events - Events that are not independent.

 Mutually Exclusive Events - Events that can not happen at the same time. Disjoint events.

 All Inclusive Events - Events whose union comprises the totality of the sample space.

 Independence: Two events are independent if the outcome of one does not affect or give an indication of the outcome of the other.

Properties and Important Rules of Probability

If an experiment can result in any of N different equally likely outcomes, and if exactly n of these correspond to event A, then

                                           P(A) =  n/N

1) 0< = P(A) =< 1

2) P(S) = 1, where S = sample space

3) For events which are mutually exclusive,

0 <= P(A) =<1

P(A1to An) = P(A1) + P(A2) +............+ P(An)

5) For any event A,

6) If A and B are mutually exclusive, then P(A  B) = 0

7) Addition Rule: If A and B are disjoint events, then P(A or B) = P(A) + P(B)

8) Multiplication Rule: If A and B are independent events, then P(A and B) = P(A)P(B)

Questions in probability can be tricky, and we benefit from a clear understanding of how to set up the solution to a problem (even if we can't solve  it!). Here is an example where intuition may need to be helped along a bit:

Remove all cards except aces and kings from a deck, so that only eight cards remain, of which four are aces and four are kings. From this abbreviated deck, deal two cards to a friend. If he looks at his card and announces (truthfully) that his hand contains an ace, what is the probability that both his cards are aces? If he announces instead that one of his cards is the ace of spades, what is the probability then that both his cards are aces? Are these two probabilities the same?" Probability theory provides the tools to organize our thinking about how to set up calculations like this. It does this by separating out the two important ingredients, namely events (which are collections of possible outcomes) and probabilities (which are numbers assigned to events). 

This separation into two logically distinct camps is the key which lets us think clearly about such problems. For example, in the rst case above, we ask \which outcomes make such an event possible?". Once this has been done we then figure out how to assign a probability to the event (for this example it is just a ratio of integers, but often it is more complicated).

First case: there are 28 possible `hands' that can be dealt (choose 2 cards out of 8). Out of these 28 hands, exactly 6 contain no aces (choose 2 cards out of 4). Hence 28-6=22 contain at least one ace. Our friend tells us he has an ace, hence he has been dealt one of these 22 hands. Out of these exactly 6 contain two aces (again choose 2 out of 4). Therefore he has a probability of 6/22=3/11 of having two aces.

Second case: one of his cards is the ace of spades. There are 7 possibilities for the other card, out of which 3 will yield a hand with 2 aces. Thus the probability is 3/7.

Sample Space

The basic setting for a probability model is the random experiment or random trial. This is your mental model of what is going on. In our previous example this would be the dealer passing over two cards to your friend. 

Defi nition : The sample space S is the set of all possible outcomes of the random experiment.

Depending on the random experiment, S may be fi nite, countably infi nite  or uncountably infi nite. For a random coin toss, S = fH; Tg, so S = 2. For our card example, S = 28, and consists of all possible unordered pairs of cards, eg (Ace of Hearts, King of Spades) etc. But note that you have some choice here: you could decide to include the order in which two cards are dealt. Your sample space would then be twice as large, and would include both (Ace of Hearts, King of Spades) and (King of Spades, Ace of Hearts). Both of these are valid sample spaces for the experiment. So you get the fi rst hint that there is some artistry in probability theory! namely how to choose the `best' sample space.

Events

An event is a collection of possible outcomes of a random experiment. Usually write A;B; ......... to denote events. So an event A is a subset of S, the sample space. Usually an event contains the set of outcomes which make the answer to a question `Yes'. Saying `the outcome is in A' is the same as saying `the event A is true'. For the first question in our card example, one event of interest is that both cards are aces. This event is the collection of all outcomes which make it true, namely the 6 hands with two aces. There are two special events: the whole sample space S is called the certain or the sure event. The empty set ; is the null event.

Example:

Pia is thirty-one years old, single, outspoken, and smart. She was a philosophy major. When a student, she was an ardent supporter of Native American rights, and she picketed a department store that had no facilities for nursing mothers. Rank the following statements in order of probability from 1 (most probable) to 6 (least probable). 

___(a) Pia is an active feminist.

___(b) Pia is a bank teller.

___(c) Pia works in a small bookstore.

___(d) Pia is a bank teller and an active feminist.

___(e) Pia is a bank teller and an active feminist who takes yoga classes.

___(f) Pia works in a small bookstore and is an active feminist who takes yoga classes.

This is a famous example, first studied empirically by the psychologists Amos Tversky and Daniel Kahneman. They found that very many people think that, given the whole story: The most probable description is (f) Pia works in a small bookstore and is an active feminist who takes yoga classes.

In fact, they rank the possibilities something like this, from most probable to least probable: (f), (e), (d), (a), (c), (b).But just look at the logical consequence rule on page 60. Since, for example, (f) logically entails (a) and (b), (a) and (b) must be more probable than (f).

Some readers of Tversky and Kahneman conclude that we human beings are irrational, because so many of us come up with the wrong probability orderings. But perhaps people are merely careless!

Perhaps most of us do not attend closely to the exact wording of the question, "Which of statements (aHf) are more probable, that is have the highest probability." Instead we think, "Which is the most useful, instructive, and likely to be true thing to say about Pia?"

When we are asked a question, most of us want to be informative, useful, or interesting, We don't necessarily want simply to say what is most probable, in the strict sense of having the highest probability. For example, suppose I ask you whether you think the rate of inflation next year will be (a) less than 3%, (b) between 3% and 4%, or (c) greater than 4%. You could reply, (a)-or-(b)-or-(c). You would certainly be right! That would be the answer with the highest probability. But it would be totally uninformative. You could reply, (b)-or-(c). That is more probable than simply (b), or simply (c), assuming that both are possible (thanks to additivity). But that is a less interesting and less useful answer than (c), or (b), by itself. Perhaps what many people do, when they look at example above, is to form a character analysis of Pia, and then make an interesting guess about what she is doing nowadays.

If that is what is happening, then people who said it was most probable that Pia works in a small bookstore and is an active feminist who takes yoga classes, are not irrational. They are just answering the wrong question-but maybe answering a more useful question than the one that was asked.

Example 1

There are three pink pencils, two blue pencils, and one green pencil. If one pencil is picked

randomly, what is the theoretical probability it will be blue?

• Find the total number of possible outcomes, that is, the total number of pencils. 3 + 2 + 1 = 6

• Find the number of specified outcomes, that is, how many pencils are blue? 2

• Find the theoretical probability. P(blue pencil) = 2/6 = 1/3.

Example 2

On the spinner, what is the probability of spinning an A or a B?

The probability of an A is 1/4. The probability of a B is 1/4. Add the two probabilities for the combined total.

1/4 + 1/4 = 2/4 = 1/2

P(A or B) = 1/2.