Recent works have called into question whether p-curve can reliably assess the presence of "evidential value" within a set of studies. To examine an as-yet unexplored issue, we examined the method used to identify p-values for inclusion in a p-curve analysis. We developed iterated p-curve analysis (IPA), which calculates and p-curves every permutation for a set of reported p-values, and applied it to the data reported in several published p-curve analyses. Specifically, we investigated two phenomena for which p-curves have been used to evaluate the presence of evidential value: the power pose and the hypothalamic-pituitary-adrenal (HPA) reactivity debates. The iterated p-curve analyses revealed that the p-curve method fails to provide reliable estimates or reproducible conclusions. We conclude that p-curve should not be used to make conclusions regarding the presence or absence of evidence for a specific phenomenon.
Copyright: © 2024 Montoya et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.