# Learning with Ensembles of Randomized Trees : New Insights

@inproceedings{Pisetta2010LearningWE, title={Learning with Ensembles of Randomized Trees : New Insights}, author={Vincent Pisetta and Pierre-Emmanuel Jouve and D. Zighed}, booktitle={ECML/PKDD}, year={2010} }

Ensembles of randomized trees such as Random Forests are among the most popular tools used in machine learning and data mining. Such algorithms work by introducing randomness in the induction of several decision trees before employing a voting scheme to give a prediction for unseen instances. In this paper, randomized trees ensembles are studied in the point of view of the basis functions they induce. We point out a connection with kernel target alignment, a measure of kernel quality, which… Expand

#### 5 Citations

Improvement of randomized ensembles of trees for supervised learning in very high dimension

- Mathematics
- 2011

Tree-based ensemble methods, such as random forests and extremely randomized trees, are methods of choice for handling high dimensional problems. One important drawback of these methods however is… Expand

On Oblique Random Forests

- Mathematics, Computer Science
- ECML/PKDD
- 2011

This work proposes to employ "oblique" random forests (oRF) built from multivariate trees which explicitly learn optimal split directions at internal nodes using linear discriminative models, rather than using random coefficients as the original oRF. Expand

Ensemble Classification and Regression-Recent Developments, Applications and Future Directions [Review Article]

- Computer Science
- IEEE Computational Intelligence Magazine
- 2016

This paper reviews traditional as well as state-of-the-art ensemble methods and thus can serve as an extensive summary for practitioners and beginners. Expand

An Ensemble Learning Based Framework for Traditional Chinese Medicine Data Analysis with ICD-10 Labels

- Medicine, Computer Science
- TheScientificWorldJournal
- 2015

The proposed ensemble model can effectively model the implied knowledge and experience in historic clinical data records and the computational cost of training a set of base learners is relatively low. Expand

Artificial Intelligence in Stochastic Multiple Criteria Decision Making

- Computer Science
- 2017

This chapter presents the concept of stochastic multiple criteria decision making (MCDM) method to solve complex ranking decision problems and different ways of improvements of these complex systems are proposed. Expand

#### References

SHOWING 1-10 OF 37 REFERENCES

Extremely randomized trees

- Mathematics, Computer Science
- Machine Learning
- 2006

A new tree-based ensemble method for supervised classification and regression problems that consists of randomizing strongly both attribute and cut-point choice while splitting a tree node and builds totally randomized trees whose structures are independent of the output values of the learning sample. Expand

An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization

- Mathematics, Computer Science
- Machine Learning
- 2004

The experiments show that in situations with little or no classification noise, randomization is competitive with (and perhaps slightly superior to) bagging but not as accurate as boosting, and sometimes better than randomization. Expand

Random Forests

- Mathematics, Computer Science
- Machine Learning
- 2004

Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the forest, and are also applicable to regression. Expand

PERT – Perfect Random Tree Ensembles

- 2001

Ensemble classifiers originated in the machine learning community. They work by fitting many individual classifiers and combining them by weighted or unweighted voting. The ensemble classifier is… Expand

Support Vector Machinery for Infinite Ensemble Learning

- Computer Science, Mathematics
- J. Mach. Learn. Res.
- 2008

An infinite ensemble learning framework based on the support vector machine (SVM) is formulated that can output an infinite and nonsparse ensemble through embedding infinitely many hypotheses into an SVM kernel and is used to derive two novel kernels, the stump kernel and the perceptron kernel. Expand

Experiments with a New Boosting Algorithm

- Computer Science
- ICML
- 1996

This paper describes experiments carried out to assess how well AdaBoost with and without pseudo-loss, performs on real learning problems and compared boosting to Breiman's "bagging" method when used to aggregate various classifiers. Expand

SOME INFINITY THEORY FOR PREDICTOR ENSEMBLES

- Mathematics
- 2000

To dispel some of the mystery about what makes tree ensembles work, they are looked at in distribution space i.e. the limit case of "infinite" sample size. It is shown that the simplest kind of trees… Expand

Spectrum of Variable-Random Trees

- Mathematics, Computer Science
- J. Artif. Intell. Res.
- 2008

In this paper, we show that a continuous spectrum of randomisation exists, in which most existing tree randomisations are only operating around the two ends of the spectrum. That leaves a huge part… Expand

Consistency of Random Forests and Other Averaging Classifiers

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2008

A number of theorems are given that establish the universal consistency of averaging rules, and it is shown that some popular classifiers, including one suggested by Breiman, are not universally consistent. Expand

Boosting as a Regularized Path to a Maximum Margin Classifier

- Mathematics, Computer Science
- J. Mach. Learn. Res.
- 2004

It is built on recent work by Efron et al. to show that boosting approximately (and in some cases exactly) minimizes its loss criterion with an l1 constraint on the coefficient vector, and shows that as the constraint is relaxed the solution converges (in the separable case) to an "l1-optimal" separating hyper-plane. Expand