An Introduction to Probability Theory

Since quantum measurements are probabilistic, it is useful to review some ideas in probability theory. The collection of all possible results of a measurement, for a physical system, is called a SAMPLE  space.

Example: 

Consider flipping a coin, there are many ways of characterizing the physical state of the system. Let’s ignore what the coin does before it lands on the table. We could characterize it’s state by the location of each of its approximately (10^23) atoms, or we could specify the orientation of a given point on the coin with respect to an axis. Thus,  sample space not only depends on the qualities of the system, but on the type of measurements we choose to make. It is common to choose heads (H) or tails (T) to characterize the physical state of the coin.  For that case, the sample space are the two possible outcomes H or T.

Suppose we have two coins, again we define a measurement as the determination of whether the coins are in the H or T states.  According to this definition we allow the following outcomes

HH
HT
TH
TT

Each of these outcomes is then a member of the sample space.

Question: Five coins are tossed. How many points are in the sample space if we define measurements as above?

Question: In addition to measuring for H or T,  one also measures it’s electrical conductivity, and thermal expansion constant. How many points are there in this sample space ?

We use the symbol S to specify a sample space. An event E is a set of sample points. So for the sample space of two coins, possible events could be

{HH}
{HT}
{TH}
{TT}

but we could also have the following events (sets of points in S)

{HH,TT}, 
{HH,TH},
{HH,HT},
{TT,TH},
{TT,HT},
{TT,HH},

etc.

{HH,HT,TH,TT}

However there are restrictions on the sets that are allowed to be events (not all sets in S are events). Let’s call the family of allowed events  F.  It should have  the following properties.

(i) S belongs in F
(ii) to F belongs the complement of any set E  that belongs in F
(iii) to F belongs the union of any sequence (Subscript[E, 1]), (Subscript[E, 2]) ... that are in F.

We should be able to define a function for event E, called P(E), which obeys the following axioms.

Axiom 1:  P(E) ⩾ 0
Axiom 2:  P(S) =1
Axiom 3: For any sequence of events  (Subscript[E, 1]) , (Subscript[E, 2]) ... (Subscript[E, n]) that are mutually exclusive (disjoint)

P [ (⋃_(n=1)^∞) (Subscript[E, n])]   = (Underoverscript[  ∑, (n=1), ∞]) P[(Subscript[E, n])]

Let's call (Subscript[w, 1]), (Subscript[w, 2]) ... the points in sample space S, then we can define

(Subscript[p, 1]) = P[(Subscript[w, 1])]   (Subscript[p, 2]) = P[(Subscript[w, 2])]   etc.,  Then, for any event E

P(E) = (Underscript[∑ , i]) (Subscript[p, i]), where index i counts all sample points in event E.

If it is possible to reproduce the sample space with a number N of independent trials, then after N trials of the experiment, the  number of occurrences  f(E) of event E, gives an approximate  value for  P(E) = f(E)/N.

It is useful to define a random variable X and assign a number to each outcome in S. If only certain values of X are allowed, it is called a discrete random variable. Otherwise it is continuous.

Example I: Consider a single coin toss sample space consisting of sample points H,T.   For H we assign X=1 and for T we assign X=0. Of course this is arbitrary we could have given X other values.

If we have a fair coin toss then we should get P(X=1)=1/2,  P(X=0) = 1/2.

Example II: Consider a two coin toss. We could assign 

X=1 for HH
X=0 for TT
X=2 for HT
X=3 for TH

but X is usually assigned in a meaningful way. Perhaps X denotes the number of heads for a sample point then

X(HH)=2
X(TT)= 0
X(HT)=1
X(TH)=1                                                                                      

Notice that with this definition X can have the same value for different sample points.
For a sample space we could have several different random variables. In the example 
above X is a number that gives the number of heads. We could then define Y as the number of
tails and so

Y(HH)=0
Y(TT)=2
Y(HT)=1
Y(TH)=1

Some external, foundations of probability theory, resources

Hyperlink["http://www.zweigmedia.com/RealWorld/tutstats/frames8_1.html"]

Hyperlink["http://www.math.uah.edu/stat/prob/Experiments.html"]

Random variables and expectation values

For a random variable X, the "equation"  X(w) = a  (X=a for short ) defines an event, as it asks which points w satisfy the equality. For example, the equality X=1 is satisfied with the points (Subscript[w, 1])=HT and (Subscript[w, 2])=TH, and so the set {HT,TH} is an event. We can also state inequalities such as  X < 2, which is satisfied by the set {TT,HT,TH}. 

We define a probability function on the random variable X so that P(X=a) = (Underscript[∑ , i]) (Subscript[p, i])  where the sum is over sample points in that event.

We can also define functions of the random variable. Suppose we have two random variables in our sample space, X, Y. We define Z=X+Y, which is a new random variable so that for sample point (Subscript[w, i])

Z((Subscript[w, i])) = X((Subscript[w, i]))+Y((Subscript[w, i])).

Expressions, such as  2 X, (X^2) ... etc.  are defined in a similar manner as those of ordinary variables.

Example:  Consider three consecutive throws of a coin. Let (Subscript[X, i])= 1 if the i th throw is a head and zero
otherwise.

({{1, 2, 3,    (Subscript[X, 1]), (Subscript[X, 2]), (Subscript[X, 3]), (Subscript[X, 1])+(Subscript[X, 2])+(Subscript[X, 3]), (Subscript[X, 1])(Subscript[X, 2])(Subscript[X, 3])}, {H, H, H,  1, 1, 1, 3, 1}, {H, H, T ,  1, 1, 0, 2, 0}, {H, T, H,  1, 0, 1, 2, 0}, {T, H, H,  0, 1, 1, 2, 0}, {H, T, T,  1, 0, 0, 1, 0}, {T, H, T,  0, 1, 0, 1, 0}, {T, T, H,  0, 0, 1, 1, 0}, {T, T, T,  0, 0, 0, 0, 0}})

Let  𝕏 = {(Subscript[w, 1]),(Subscript[w, 2]), ... } be a discrete sample space with probability measure P assigning the values (Subscript[p, i]) to the sample points  (Subscript[w, i]). If X is a random variable, then the expectation value of X, denoted by E[X] or simply EX, is defined as 

X((Subscript[w, 1])) (Subscript[p, 1]) + X((Subscript[w, 2]))  (Subscript[p, 2]) + ...

We assume that this series converges to a definite value. Among physicists, the notation  ⟨X⟩ is commonly used in place of EX and we adopt use of the former in the textbook. 

Another important global property of our sample space is called the variance (σ^2). It's defined as

(σ^2) = E[ (((X-EX))^2)]

i.e.  define a new random variable  (((X-EX))^2) and take its expectation value. Notice we can simplify this expression since

(((X-EX))^2) = (X^2) - X EX - EX X + (EX^2)

and so 

E[(((X-EX))^2)]= E[(X^2)]-E[X] EX- EX E[X] + (EX^2)=  E[(X^2)]-2 (EX^2)+ (EX^2)

or

(σ^2) = E[(X^2)] - (EX^2)  

that is,  (σ^2) is the difference between the expectation value of (X^2) and the square of the expectation value EX.

Conditional probability

Up to now we have considered probability measure such as P(A) where A is a set in our sample space. According to the frequency Interpretation it is roughly equal to the number of times event A occurs in N independent trials, divided by N. Suppose we have more information than that. If we know that A must occur in B then we can define something called  P(A|B). It is the conditional probability of event A given B. 

P(A|B) =  P(A ⋂ B)/P(B)