Comparison of dimensionality reduction and feature selection for cognitive task decoding using functional connectivity.
Authors
Affiliations (3)
Affiliations (3)
- University of Illinois Urbana-Champaign, Department of Psychology, 603 E Daniel St, Champaign, IL, 61820. Electronic address: [email protected].
- University of Wisconsin-Milwaukee, Department of Psychology, 2441 E. Hartford Ave., Milwaukee, WI 53211. Electronic address: [email protected].
- University of Illinois Urbana-Champaign, Department of Psychology, 603 E Daniel St, Champaign, IL, 61820.
Abstract
Advances in functional magnetic resonance imaging (fMRI) have led to the ability to study the brain across many contexts. However, the large number of features generated by functional connectivity approaches may overfit the data. These problems can be overcome with either feature selection (FS) or dimensionality reduction (DR), which can be applied to less complex models. We utilize two open source datasets to compare the performance of DR/FS methods on cognitive task decoding using a suite of ML classifiers. While FS and DR methods have been used previously in decoding research, no systematic comparison of their performance has been undertaken. Here, we compare available methods using commonly utilized machine learning libraries to establish which methods provide the best predictive performance. We then conduct statistical tests to examine the relative contributions of DR and FS methods and classifiers on decoding accuracy. Neither DR or FS was found to be superior. However, differences were identified across datasets and tasks. In the majority of methods and datasets, a peak in predictive performance was found using a small percentage (005-.10%) of the total number of original features. Some methods perform better than the baseline method of prediction with all available features or selecting features randomly. Decoding performance with some datasets with some method exceeds that of deep learning approaches. These results suggest a "sweet spot" for the tradeoff between the retention of features and predictive accuracy.