We consider the effective field theory of gravity around black holes, and show that the coefficients of the dimension-8 operators are tightly constrained by causality considerations. Those constraints are consistent with-but tighter than-previously derived causality and positivity bounds and imply that the effects of one of the dimension-8 operators by itself cannot be observable while remaining consistent with causality. We then establish in which regime one can expect the generic dimension-8 and lower order operators to be potentially observable while preserving causality, providing a theoretical prior for future observations. We highlight the importance of "infrared causality" and show that the requirement of "asymptotic causality" or net (sub)luminality would fail to properly diagnose violations of causality.