Decision Tree

ID3

$$I(P)=-\sum _{i=1}^n{p}_i{\log }_2{p}_i$$

A higher tendency to choose an attribute containing more values

C4.5

$$\mathrm{SplitInfo}(A)=-\sum _{i=1}^n{p}_i{\log }_2{p}_i\text{where }{p}_i\in A$$

SplitInfo $\propto$ number of distinct values, penalize an attribute containing more values

CART

$$\mathrm{Gini}(P)=1-\sum _{i=1}^n{p}_i^2$$

Measurments

Lift Chart

1. Sort by predicted value then actual yes
2. Draw lift curve (cumulative actual yes)
3. The larger the area between lift curve and base line the better

Decile-wise Lift Chart

1. Sort by predicted value then actual yes
2. Draw vertical line by actual yes ($\Delta \%=\frac{100}{N}\%$)
3. Draw vertical line by decile ($\Delta \%=10\%$)
4. Compute decile mean ($\frac{\text{Actual yes in decile}}{10}$)
5. Compute global mean ($\frac{\text{Actual yes}}{N}$)
6. Draw chart ($y=\frac{\text{Decile mean}}{\text{Global mean}}$)

XLMiner

Glossaries

• Success probability cutoff $\to$ terminal node yes/no cutoff

Partitioning

• Training set
  • Create
• Validation set
  • Adjust/Fine-tune
• Test set
  • Test

Usually it is training 50%, validation 30%, test 20%

Categories Reduction

• Automatic
  1. Sort values by frequency
  2. Assign categories in descending order of frequency
  3. Remaining values will be assigned to the last category
• Manual

This might be needed, if input/output attribute has too much distinct values.