The characterization of the layer properties of multilayered structures has attracted research interest owing to advanced applications in fields of atom-based sensors, ultra-narrow optical filters, and composite films. Here, a robust non-destructive multipath interferometry method is proposed to characterize the features of a thin cell with a borosilicate glass-rubidium-borosilicate glass sandwiched structure using a femtosecond optical frequency comb. The multipath interference method serves as a powerful tool for identification of the layer number and physical thickness of a three-layered structure. Moreover, the global distribution map is obtained by scanning the entire region. Furthermore, the amplitude of sub-Doppler reflection spectra of the rubidium D2 line is confirmed at different target points to validate this method. This result promotes the development of thin-cell-based atomic devices with strong light-matter interaction at atomic scales.