[Solomonov Seminar] 170. Solomonov seminar
marko.grobelnik at ijs.si
Tue May 30 02:17:04 CEST 2006
Vabim vas na 170. Solomonov seminar, ki bo danes, v torek 30. maja
ob 13:00 uri v Veliki predavalnici IJS.
Tokrat bo predaval nas gost Rich Caruana iz Cornel University v ZDA,
ki je priznan strokovnjak s podrocja strojnega ucenja (Machine Learning).
Predaval o temi, ki bo zanimiva za sirok krog poslusalcev, namrec o tem
katero analiticno metodo izbrati v kateri situaciji in kako evaluirati
rezultate na pravi nacin.
Rich Caruana, Cornel University:
Which Supervised Learning Method Works Best for What?
An Empirical Comparison of Learning Methods and Metrics.
Decision trees are intelligible, but do they perform well enough that you
should use them? Have SVMs replaced neural nets, or are neural nets still
best for regression, and SVMs best for classification? Boosting maximizes
margins similar to SVMs, but can boosting compete with SVMs? And if it does
compete, is it better to boost weak models, as theory might suggest, or to
boost stronger models? Bagging is simpler than boosting -- how well does
bagging stack up against boosting? Breiman said Random Forests are better
than bagging and as good as boosting. Was he right? And what about old
friends like logistic regression, KNN, and naive bayes? Should they be relegated
to the history books, or do they still fill important niches?
In this talk we compare the performance of ten supervised learning methods
on nine criteria: Accuracy, F-score, Lift, Precision/Recall Break-Even Point,
Area under the ROC, Average Precision, Squared Error, Cross-Entropy,
and Probability Calibration. The results show that no one learning method
does it all, but some methods can be "repaired" so that they do very well across
all performance metrics. In particular, we show how to obtain the best probabilities
from max margin methods such as SVMs and boosting via Platt's Method and
isotonic regression. We then describe a new ensemble method that combines
select models from these ten learning methods to yield much better performance.
Although these ensembles perform extremely well, they are too complex for many
applications. We'll describe what we're doing to try to fix that. Finally, if time
permits, we'll discuss how the nine performance metrics relate to each other,
and which of them you probably should (or shouldn't) use.
During this talk I'll briefly describe the learning methods and performance metrics
to help make the lecture accessible to non-specialists in machine learning.
More information about the Solomonov-seminar