Comparison of combining methods using Extreme Learning Machines under small sample scenario

Making accurate predictions is a difficult task that is encountered throughout many research domains. In certain cases, the number of available samples is so scarce that providing reliable estimates is a challenging problem. In this paper, we are interested in giving as accurate predictions as possible based on the Extreme Learning Machine type of a neural network in small sample data scenarios. Most of the Extreme Learning Machine literature is focused on choosing a particular model from a pool of candidates, but such approach usually ignores model selection uncertainty and has inferior performance compared to combining methods. We empirically examine several model selection criteria coupled with new model combining approaches that were recently proposed. The results obtained indicate that a careful choice among the combinations must be performed in order to have the most accurate and stable predictions.