EEG data, that are too high-dimensional for the application of standard regression methods. An .... do not need to be sp
Carolin Strobl, James Malley and Gerhard Tutz
An Introduction to Recursive Partitioning
Technical Report Number 55, 2009 Department of Statistics University of Munich http://www.stat.uni-muenchen.de
An Introduction to Recursive Partitioning: Rationale, Application and Characteristics of Classification and Regression Trees, Bagging and Random Forests
Carolin Strobl
James Malley
Department of Statistics
Center for Information Technology
Ludwig-Maximilians-Universit¨at
National Institutes of Health
Munich, Germany
Bethesda, MD, USA
Gerhard Tutz Department of Statistics Ludwig-Maximilians-Universit¨at Munich, Germany
Abstract Recursive partitioning methods have become popular and widely used tools for nonparametric regression and classification in many scientific fields. Especially random forests, that can deal with large numbers of predictor variables even in the presence of complex interactions, have been applied successfully in genetics, clinical medicine and bioinformatics within the past few years. High dimensional problems are common not only in genetics, but also in some areas of psychological research, where only few subjects can be measured due to time or cost constraints, yet a large amount of ) • Select some subjects. (Otherwise fitting will take a while, because all combinations of subjects need to be compared for parameter instabilities in their regression models.) > dat_sleep dat_sleep$Subject mymob 10.227
2 gene_11676 p = 0.27 ≤ 3.725 3 n=0 y = (0.429, 0.571)
1 brain_pH p = 0.005 ≤ 6.67
5 n=0 y = (0.867, 0.133)
> 3.725
≤ 6.16
4 n=0 y = (0.1, 0.9)
> 6.67
2 gene_9489 p = 0.071
3 n=0 y = (0.867, 0.133)
5 gene_13669 p = 0.124
> 6.16
≤ 6.014
4 n=0 y = (0.375, 0.625)
1 gene_2807 p = 0.002
6 n=0 y = (0, 1)
> 6.014 7 n=0 y = (0.375, 0.625)
1 simulated_gene_1 p = 0.003
≤ 6.742
> 6.742
2 n=0 y = (0.786, 0.214)
3 simulated_gene_1 p = 0.001 ≤ 12.279 4 gene_5236 p = 0.026 ≤ 4.876 5 n=0 y = (0.125, 0.875)
≤ 12.413
> 12.279 7 n=0 y = (0.75, 0.25)
> 4.876
≤ 8.204 3 n=0 y = (0.667, 0.333)
6 n=0 y = (0, 1)
> 12.413
2 gene_1440 p = 0.002
5 gene_8717 p = 0.026
> 8.204
≤ 8.023
4 n=0 y = (0.071, 0.929)
6 n=0 y = (1, 0)
> 8.023 7 n=0 y = (0.625, 0.375)
• Compute and plot the permutation importance of each predictor variable. > set.seed(2908) > myvarimp barplot(myvarimp[90:100], space=0.75, xlim=c(0,0.035), +
names.arg=rownames(myvarimp)[90:100], horiz=TRUE, cex.names=0.45,
+
cex=0.45, las=1)
gene_17678
gene_5491
gene_7193
gene_8674
gene_4087
gene_11676
gene_12588
gene_21180
gene_9569
simulated_gene_2
gene_10430
0.000
0.005
0.010
0.015
0.020
0.025
0.030
0.035
(Only a few genes are displayed here to save space. All but the first plot options are only for aesthetics.) • Prediction in terms of the predicted response class or the predicted class probabilities for some selected subjects.
An Introduction to Recursive Partitioning 6
> subjects y y_hat y_hat y_hat_oob sum(dat_genes$status==y_hat)/nrow(dat_genes) [1] 0.9016393 > sum(dat_genes$status==y_hat_oob)/nrow(dat_genes) [1] 0.6721311 > table(dat_genes$status, y_hat) y_hat Bipolar disorder Healthy control Bipolar disorder
28
2
Healthy control
4
27
An Introduction to Recursive Partitioning 7
> table(dat_genes$status, y_hat_oob) y_hat_oob Bipolar disorder Healthy control Bipolar disorder
20
10
Healthy control
10
21
References Hothorn, T., K. Hornik, and A. Zeileis (2006). Unbiased recursive partitioning: A conditional inference framework. Journal of Computational and Graphical Statistics 15 (3), 651–674. Leisch, F. (2002). Sweave: Dynamic generation of statistical reports. In W. H¨ardle and B. R¨onz (Eds.), Proceedings in Computational Statistics, Heidelberg, pp. 575–580. Physika Verlag. Strobl, C., A.-L. Boulesteix, A. Zeileis, and T. Hothorn (2007). Bias in random forest variable importance measures: Illustrations, sources and a solution. BMC Bioinformatics 8:25.
