Concept - Splitting Continuous Attributes

Splitting Continuous Attributes (decision trees)

Two options (s.317):

1. Discretize in advance real-valued attributes into ranges (big, medium, small). Example s.318: GPS tracking (stop/walk) with speed: “Speed < 3?” partitions the sample speeds [3, 0.5, 10, 4, 0.3, 0.1, 1] into a small group and [3, 10, 4].
2. Threshold split $A<c$ vs $A\ge c$; its Concept - Information Gain is easily computed and compared with discrete splits to select the best split.

Finding the best threshold (s.320)

For each continuous feature $A$: sort examples by $A$; for each ordered neighbour pair $X_i,X_{i+1}$, if their categories differ, take the midpoint $c_i$ → candidate threshold $A<c_i$. Example (Temp, from the 14 Play Tennis days, s.319-320): sorted 64, 65, 68, 69, 70, 71, 72, 72, 75, 75, 80, 81, 83, 85 → candidates 64.5, 66.5, 70.5, 72, 77.5, 80.5, 84.

Con noted in s.328: trees lose information when categorizing continuous variables.

Related

Concept - Information Gain | Concept - Decision Tree

Appears in

• 03 Text Classification - Chunk 11 (s.301-330)