A set of benchmark systems is defined to compare different computational approaches for characterizing local minima, transition states, and pathways in atomic, molecular, and condensed matter systems. Comparisons between several commonly used methods are presented. The strengths and weaknesses are discussed, as well as implementation details that are important for achieving good performance. All of the benchmarks and methods are provided in an online database to make the implementation details available and the results reproducible. While this paper provides a snapshot of the benchmark results, the online framework is structured to be dynamic and incorporate new methods and codes as they are developed.