\documentclass[12pt]{article} \usepackage{Math203} \usepackage{amsmath,amssymb} \usepackage{hyperref} \usepackage{color} \usepackage{xcolor} \usepackage{enumitem} \pagestyle{plain} \renewcommand{\labelenumi}{\arabic{enumi}.} \renewcommand{\labelenumii}{\arabic{enumi}.\arabic{enumii}} \renewcommand{\labelenumiii}{\arabic{enumi}.\arabic{enumii}.\arabic{enumiii}} \setlength{\topmargin}{-0.75 in} \setlength{\textheight}{9.5 in} \parindent0in \newenvironment{packed_enum}{ \begin{enumerate} \setlength{\itemsep}{12pt} \setlength{\parskip}{0pt} \setlength{\parsep}{0pt} \setlength{\topsep}{0pt} }{\end{enumerate}} \renewcommand{\labelitemii}{$\blacktriangleright$} \def\ctitle{Math 203: Principles of Statistics 1} \def\ttitle{Fall 2024} \def\stitle{The Binomial Distribution} \makeatletter \def\@maketitle{% \newpage \null \begin{center}% {\textsclarge{\ctitle} \par}% \medskip {\textsclarge{\ttitle} \par}% \medskip {\textsc{\stitle} \par} \end{center}% \par \vskip 1.5em} \makeatother \begin{document} <>= myrnd <- function(x,d) {formatC(x, format="f", digits=d)} library(shape) library(diagram) @ \maketitle The Binomial distribution is a probability distribution for a random variable derived from sequences of \textit{binary trials}, that is, experiments that result in a numerical outcome recorded as 0 or 1. We refer to `successes' (outcome is 1) and `failures' (outcome is 0). Each trial has `success probability' $p$, a numerical value between zero and one, that is, the probability that the outcome is 1 is $p$, and the probability that the outcome is 0 is $q=1-p$, \bigskip \textbf{Experimental set up:} We consider the experiment involving a sequence of $n$ identical binary trials with success probability $p$ whose outcomes are independent. Let $A_i$ represent the event that the $i$th trial results in a success. One possible sequence for $n=8$ is \[ 01010111 \] corresponding to the event \[ A_1^c \cap A_2 \cap A_3^c \cap A_4 \cap A_5^c \cap A_6 \cap A_7 \cap A_8 \] which under independence of the individual trials has probability \[ (1-p) \times p \times (1-p) \times p \times (1-p) \times p \times p \times p = p^5 (1-p)^3 = p^5 q^3. \] There are $2^8 = \Sexpr{2^8}$ binary sequences of length eight in total. \bigskip \textbf{Random variable:} To define the random variable, $Y$, we record the \textbf{total number} of successes in the sequence. If $0 < p < 1$, the set of possible values $Y$ can take is \[ \calY = \{0,1,2,\ldots,n\}. \] All of these values will carry some positive probability. \bigskip \textbf{Probability distribution:} This definition for $Y$ allows us to compute the probability that should be attached to each of the possible values that $Y$ can take. We consider events of the form $(Y=y)$ for numerical values $y=0,1,2,\ldots,n$, and observe that \[ (Y = y) \quad \text{means} \quad ``\text{$y$ successes in $n$ trials}" \] Consider a sequence that results in $y$ successes and $n-y$ failures: for example, if $n=8$ and $y=3$, we might have \[ \begin{array}{c} 10101000\\ 01010001\\ 11100000 \end{array} \] and so on. All such sequences are equally likely under the `identical and independent' assumption. The probability of any such sequence is $p^y q^{n-y}$, for $y \in \{0,1,2,\ldots,n\}$; that is, in the examples above \[ \begin{array}{cc} 10101000 & pqpqpqqq\\ 01010001 & qpqpqqqp\\ 11100000 & pppqqqqq \end{array} \] etc. all yield $p^3 q^5$. For the \textbf{total} number of successes, we do not care about the order of the outcomes; we therefore must count the number of all such sequences, as the individual sequences partition the event of interest, which in the example is $(Y=3)$. By previous combinatorial arguments, we see that the number of such sequences is \[ C_y^n = \dbinom{n}{y} = \frac{n!}{y! (n-y)!} = \frac{8!}{3! 5!} = \Sexpr{choose(8,3)}. \] Therefore we have that \[ P(Y=y) = \dbinom{n}{y} p^y q^{n-y} \qquad y \in \{0,1,2,\ldots,n\} \] and $P(Y=y)=0$ for all other $y$. We therefore have that the \textbf{probability mass function} (pmf) for the Binomial distribution is \[ p(y) = \dbinom{n}{y} p^y q^{n-y} \qquad y \in \{0,1,2,\ldots,n\} \] with $p(y)=0$ otherwise. The pmf is the mathematical function that describes how probability is distributed across the possible values of the random variable. We write \[ Y \sim Binomial(n,p). \] \bigskip \textbf{Numerical calculations:} Numerical calculations for the binomial distribution can be performed in \texttt{R}. <>= n<-8 p<-0.4 q<-1-p y<-3 prob<-choose(n,y)*(p^y)*((1-p)^(n-y)) #choose(.,.) computes C_n^r. print(paste('P(Y=3) =',prob)) @ The calculation can also be performed using the specially written function \texttt{dbinom} <>= print(paste('P(Y=3) =',dbinom(y,n,p))) @ We can compute the complete list of values for the pmf evaluated at $y=0,1,2,\ldots,n$. <>= y<-0:n; probs<-dbinom(y,n,p) print(cbind(y,probs)) @ We can plot the pmf as a function of $y$. <>= par(mar=c(4,4,2,2)) plot(y,probs,pch=19,ylab='p(y)',ylim=range(0,0.4),xaxt='n');axis(1,at=0:8) @ \medskip \textbf{Connection to samples:} We can simulate data mimicking the experiment. For example, we can generate a matrix of sequences of binary trials, and then compute $Y$ for each sequence; here for $N=5$ sequences <>= set.seed(3) N<-5 X<-matrix(rbinom(n*N,1,p),nrow=5,ncol=n) #n*N is the total number of trials rownames(X)<-paste('Sequence',1:5); colnames(X)<-paste(1:n) Y<-rowSums(X) cbind(X,Y) @ Hence the simulated $Y$ values are \Sexpr{Y}. We can also simulate $Y$ directly using \texttt{rbinom}. Here are $N=10$ values <>= N<-10;rbinom(N,n,p) @ Suppose we simulate a very large sample $N=10000$, and draw a density histogram <>= N<-10000 Y<-rbinom(N,n,p) par(mar=c(4,4,0,0)) hist(Y,breaks=0:9,right=FALSE,include.lowest=TRUE,main='', xaxt='n',freq=FALSE,ylim=range(0,0.4)) axis(1,at=c(0:8)) box() points(y,probs,pch=19,col='red') legend(6,0.3,c('p(y)'),pch=19,col='red') @ The pmf values are superimposed in red. The pmf can therefore be interpreted as the limiting values of the density histogram heights in a very large sample. \bigskip In the sample of \Sexpr{N} from the $Binomial(\Sexpr{n},\Sexpr{p})$ distribution, we have the following counts of the different possible values. <>= table(Y) @ The relative frequencies are <>= table(Y)/N @ which closely match the pmf values <>= round(cbind(table(Y)/N,probs),6) @ \textbf{Expectation and Variance:} If $Y\sim Binomial(n,p)$, then the (population) mean (or expectation) and variance are \[ E(Y) = np \qquad Var(Y) = npq \] <>= c(n*p,n*p*q) #Expectation, Variance @ That is, the expectation is $\Sexpr{n*p}$ and the variance is $\Sexpr{n*p*(1-p)}$. These values describe aspects of the distribution, and the function $p(y)$. In the very large sample of $N=100000$ values, we have sample mean and sample variance which are very close to the population expectation and variance <>= mean(Y) var(Y) @ \end{document}