Understanding the relationship between the sequence and binding energy in peptide-protein interactions is an important challenge in chemical biology. A prominent example is ubiquitin interacting motifs (UIMs), which are short peptide sequences that recognize ubiquitin and which bind individual ubiquitin proteins with a weak affinity. Though the sequence characteristics of UIMs are well understood, the relationship between the sequence and ubiquitin binding affinity has not yet been fully characterized. Herein, we study the first UIM of Vps27 as a model system. Using an experimental alanine scan, we were able to rank the relative contribution of each hydrophobic residue of this UIM to ubiquitin binding. These results were correlated with AlphaFold displacement studies, in which AlphaFold is used to predict the stronger binder by presenting a target protein with two potential peptide ligands. We demonstrate that by generating large numbers of models and using the consensus bound-state AlphaFold competition experiments can be sensitive to single-residue variations. We furthermore show that to fully recapitulate the binding trends observed for ubiquitin, it is necessary to screen AlphaFold models that incorporate a "decoy" binding site to prevent the displaced peptide from interfering with the actual binding site. Overall, it is shown that AlphaFold can be used as a powerful tool for peptide binder design and that when large ensembles of models are used, AlphaFold predictions can be sensitive to very small energetic changes arising from single-residue alterations to a binder.