Random Forest

To begin talking about random forest we need to cover bagging.

Bagging

Bootsrap Aggregating or Bagging is done by training \(T\) trees where for each tree it is trained by some dataset \(X' \subseteq X\) where \(X'\) is generated by sampling \(n'\) samples from \(X\) with replacement.

Random Forest

Now, random forest takes bagging and add another randomness to the features used by each tree. Here, each tree takes \(d' < d\) features randomly sampled without replacement from the features.

• For classification use \(d' \approx \sqrt{d}\).

• For regression use \(d' \approx d/3\).