[Solomonov Seminar] 170. Solomonov seminar

[Marko Grobelnik]

Vabim vas na 170. Solomonov seminar, ki bo danes, v torek 30. maja 
ob 13:00 uri v Veliki predavalnici IJS. 

Tokrat bo predaval nas gost Rich Caruana iz Cornel University v ZDA,
ki je priznan strokovnjak s podrocja strojnega ucenja (Machine Learning).
Predaval o temi, ki bo zanimiva za sirok krog poslusalcev, namrec o tem
katero analiticno metodo izbrati v kateri situaciji in kako evaluirati
rezultate na pravi nacin.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Rich Caruana, Cornel University:
   Which Supervised Learning Method Works Best for What? 
   An Empirical Comparison of Learning Methods and Metrics.

Decision trees are intelligible, but do they perform well enough that you 
should use them?  Have SVMs replaced neural nets, or are neural nets still 
best for regression, and SVMs best for classification? Boosting maximizes 
margins similar to SVMs, but can boosting compete with SVMs?  And if it does 
compete, is it better to boost weak models, as theory might suggest, or to 
boost stronger models?  Bagging is simpler than boosting -- how well does 
bagging stack up against boosting?  Breiman said Random Forests are better 
than bagging and as good as boosting.  Was he right?  And what about old 
friends like logistic regression, KNN, and naive bayes?  Should they be relegated 
to the history books, or do they still fill important niches?
 
In this talk we compare the performance of ten supervised learning methods 
on nine criteria: Accuracy, F-score, Lift, Precision/Recall Break-Even Point, 
Area under the ROC, Average Precision, Squared Error, Cross-Entropy, 
and Probability Calibration.  The results show that no one learning method 
does it all, but some methods can be "repaired" so that they do very well across 
all performance metrics. In particular, we show how to obtain the best probabilities 
from max margin methods such as SVMs and boosting via Platt's Method and 
isotonic regression.  We then describe a new ensemble method that combines 
select models from these ten learning methods to yield much better performance.  
Although these ensembles perform extremely well, they are too complex for many 
applications.  We'll describe what we're doing to try to fix that.  Finally, if time 
permits, we'll discuss how the nine performance metrics relate to each other, 
and which of them you probably should (or shouldn't) use.
 
During this talk I'll briefly describe the learning methods and performance metrics 
to help make the lecture accessible to non-specialists in machine learning.