vault backup: 2025-03-16 18:59:42

AI & Data Mining/Week 6/Lecture 11 - ID3.md

@@ -1,6 +1,7 @@
 # Logarithms
 
 $log_2X$ used for generating decision trees
+
 - Power to which we have to raise 2 to get X
 - When using, X will be probability between 0 and 1
 - log of probability is always negative
@@ -48,19 +49,23 @@ $log_2X$ used for generating decision trees
 	- Given probability distribution, info required to predict an event is the distributions entropy
 	- Entropy gives the information required in bits
 
-# $I(p_1,p_2,...,p_n)=-p_{1}\log_{2}p_1 -p_{2}\log_{2}p_2 ... -p_{n}\log_{2}p_n$
- Where n = number of classes, and $p_1 + p_2 + ... p_{n} = 1$
+# $I(p_1,p_2,…,p_n)=-p_{1}\log_{2}p_1 -p_{2}\log_{2}p_2 … -p_{n}\log_{2}p_n$
+
+ Where n = number of classes, and $p_1 + p_2 + … p_{n} = 1$
 Minus signs included since output must be positive
 
 ### Expected Information for Outlook
 
 - Outlook = Sunny
+
 # $info([2,3]) = I(\frac{2}{5},\frac{3}{5}) = -\frac{2}{5}\log_2(\frac{2}{5}) - \frac{3}{5}\log_2(\frac{3}{5}) = 0.971 bits$
 
 - Outlook = Overcast
+
 # $info([4,0]) = I(\frac{4}{4},\frac{0}{4}) = -1\log_2(1) -0\log_2(0) = 0 bits$
 
 - Outlook = Rainy
+
 # $info([3,2]) = I(\frac{3}{5},\frac{2}{5}) = -\frac{3}{5}\log_2(\frac{3}{5}) - \frac{2}{5}\log_2(\frac{2}{5}) = 0.693 bits$
 
 ### Computing Information Gain

AI & Data Mining/Week 6/Lecture 12 - Decision Trees (ID3).md

@@ -1,2 +1 @@
 ![](Pasted%20image%2020241025132339.png)
-