Definition of probability
Axioms of probability theory
Definition of probability:
The probability is the concept which is hard to characterize formally. The
temptation is to define it in terms of frequency of events in repeated experiments, but,
this approach leads to circular definition, so we will define probability in terms of
axioms.
Axioms of Probability Theory:
Let S be a sample space for some experiment. For each event E ⊂ S, we assume that
a number P(E) is defined and satisfies the following three axioms:
Axiom 1: 0  P( S )  1 .
Axiom 2: P( S )  1 .
Axiom 3: If {Ei } are mutually exclusive ( Ei  E j  
i  j ) , then
 
 
PU Ei    P( Ei ) . We call P( A) the probability of E .
 i 1  i 1
Theorem: P ( )  0
Theorem: Let A1 , A2 , A3 ,..., An be mutually exclusive (but finite). Then,
   
PU Ai    P( Ai ) .
 i 1  i 1
A  B , then P( A)  P( B )
Theorem: Inclusion /Exclusion principle: For any pair of events A, B
P( A  B )  P( A)  P( B )  P( A  B )
Theorem: Union of events bound. For any pair of events A, B
P( A  B )  P( A)  P( B )
EXAMPLES:
1- Consider flipping a coin and recording the outcome. Then S = {H, T}. Need
a probability. Assume fair coin. Then
P({H}) = 1/2
P({T}) = 1/2
P({H, T}) = 1
Note that P({H} ∪ {T}) = 1 = 1/2 + 1/2 = P({H}) + P({T}).
This is not the only probability possible for this experiment/sample space. Could have
an unfair coin.
P({H}) = p,
P({T}) = 1 − p,
P({H, T}) = 1
2- Rolling a die. Assuming that the die is fair, P({1}) = P({2}) = . . . = P({6})
= 1/6.
Probability of rolling an even number is
P({2, 4, 6}) = P({2}) + P({4}) + P({6}) = 1/6 + 1/6 + 1/6 = 1/2
Probability of rolling an odd number is
P({1, 3, 5}) = P({1}) + P({3}) + P({5}) = 1/6 + 1/6 + 1/6 = 1/2
Probability of rolling a prime number is
P({2, 3, 5}) = P({2}) + P({3}) + P({5}) = 1/6 + 1/6 + 1/6 = 1/2
Probability of rolling a number less than 5 is
P({1,2,3, 4}) = P({1})+P({2}) + P({3}) + P({4}) = 1/6 +1/6 + 1/6 + 1/6 = 2/3