CIS 5371 Cryptography 3. Probability & Information Theory 1 B i rules Basic l of f probability b bilit N Notation i Events: S, , E, F, ..., EF, EF, … . Pr[S]=1 Pr[]=0, Pr[S]=1, Pr[]=0 0 Pr[E] 1 Pr[EF] = Pr[E] + Pr[F] - Pr[EF], . . E S \ E , Pr[ E ] Pr [ E ] 1 EF Pr[ E ] Pr[ F ] Pr[ E F ] Pr[ F | E ] Pr[ E ] 2 B i rules Basic l of f probability b bilit A Experiment An E i can yield i ld one off n equally probable outcomes. Event E1: One of the n values occurs. Pr[E1] = 1/n Event E2: m of the n values occur. Pr[E2] = m/n Event E3: An Ace is drawn from a pack of 52 cards Pr[E3] [ ] = 4/52 = 1/13 3 B i rules Basic l of f probability b bilit Binomial Distributi on : n k P [ k succeses in n trials Pr l ] p (1 p ) n k k Bayes ' Law : Pr[ E ] Pr[ E | F ] Pr[ F | E ] Pr[ F ] Pr[F 4 Bi thd P Birthday Paradox d Let f : X Y where Y is a set of n elements Event E k , : for k pairwise distinct values x1 , x 2 , , x k the probabilit y of a collision f ( xi ) f ( x j ), occurs for some i j , is at least Birthday Paradox : If 1 / 2 then k 1.1774 n 5 I f Information ti Th Theory Entropy (Shannon) The entropy of a message source is a measure of the amount of information the source has Let L = {a1, … , an} be a language with n letters. S is a source that outputs these letters with independent probabilities Prob[a1], … , Prob[an]. The h entropy off S is: i H (S ) n i 1 1 bits Pr[ a i ] log 2 Pr[ a i ] 6 I f Information ti Th Theory Example A source S outputs a random bit. Its entropy is: 1 1 Pr[1] log 2 H ( S ) Pr[0] log 2 Pr[0] Pr[1] 1 1 (1) (1) 1 bit 2 2 7