Feature selection is a critical component in supervised machine learning classification analyses. Extraneous features introduce noise and inefficiencies into the system leading to a need for feature reduction techniques. Many feature reduction models use the end-classification results in the feature reduction process, committing a circular error. Item Response Theory (IRT) examines the characteristics of features independent of the end-classification results, and provides high levels of information regarding feature utility. A two-parameter dichotomous IRT model was used to analyze 18 features from an intensive care unit data set with 2520 cases. The classification results showed that the features selected via IRT were comparable to that using more traditional machine learning approaches. Strengths and limitations of the IRT selection protocol are discussed.