Spontaneous extrusion of a kidney stone is an extremely rare event with few reported cases. We present the unusual case of a 62-year-old female who presented with worsening of fever and flank pain over a 5 day period. Imaging revealed retroperitoneal and abdominal wall abscesses that upon evaluation of prior imaging were attributed to spontaneously extruded kidney stones. Prior reports suggest surgical exploration and extraction is the preferred treatment option for extruded renal calculi. Based on the experience presented here, conservative management may be an attractive first-line treatment option for patients with extruded renal stones and associated retroperitoneal abscesses.