vault backup: 2024-12-01 23:50:59

AI & Data Mining/Week 9/Chapter 15.md

@@ -0,0 +1,33 @@
+1)
+a) Binomial Distribution
+b) Measures dispersion of probabilities with respect to a mean average value. Each possible value of S from 0 to N, the probability of observing S correct predictions given a sample of N independent examples of true accuracy P
+
+2)
+a) (150 + 180 + 420) / (150 + 180 + 420 + 30 + 50 + 50 + 40 + 50 + 30) = 0.75
+
+# Variance of S $\sigma^2_S = N_p(1-p)$ 
+# Std Dev of S $\sigma_S = \sqrt{N_p(1-p)}$
+# Variance in F $\sigma_f = \frac{\sigma_S}{N} = \sqrt{\frac{N_p(1-p)}{N^2}} = \sqrt{\frac{p(1-p)}{N}}$
+
+# Estimate of Predictive Accuracy $\mu_f = \frac{S}{N}$
+# Successful Trials $S$
+# Number of Trials $N$
+
+
+750 Successes 1000 Trials
+S = 750 
+N = 1000
+$\mu_f$ = 0.75
+$\sqrt{(0.75 \times 0.25)/1000} = 0.0137$
+when c = 80%, (100-80)/2 = 10%, z = 1.28
+
+$\mu_f \pm z \times \sigma_f = 0.75 \pm (1.28 \times 0.0137)$
+$= 0.75 \pm 0.0175$
+
+p lies between 73.25% and 76.75%, with 80% confidence.
+
+3)
+a)
+Stratified Holdout, data split to guarantee same distribution of class values in training and test set
+b)
+Repeated Holdout, training and testing done several times with different splits. Overall estimate of predictive accuracy is average of predicted accuracy in different iteration