The probability-based search engine MASCOT has been widely used to identify peptides and proteins in shotgun proteomic research. Most subsequent quality control methods filter out ambiguous assignments according to the ion score and thresholds provided by MASCOT. On the basis of target-decoy database search strategy, we evaluated the performance of several filter methods on MASCOT search results and demonstrated that using filter boundaries on two-dimensional feature spaces, the MASCOT ion score and its relative score can improve the sensitivity of the filter process. Furthermore, using a linear combination of several characteristics of the assigned peptides, including the MASCOT scores, 15 previously employed features, and some newly introduced features, we applied a Bayesian nonparametric model to MASCOT search results and validated more correctly identified peptides in control and complex data sets than those could be validated by empirical score thresholds.