We examine four olivine-bearing regions at a variety of spatial scales with thermal infrared, visible to near-infrared, and visible imagery data to investigate the hypothesis that the relationship between olivine abundance and thermal inertia (i.e., effective particle size) can be used to infer the occurrence of olivine chemical alteration during sediment production on Mars. As in previous work, Nili Fossae and Isidis Planitia show a positive correlation between thermal inertia and olivine abundance in Thermal Emission Spectrometer (TES) and Thermal Emission Imaging System (THEMIS) data, which could be interpreted as indicating olivine chemical weathering. However, geomorphological analysis reveals that relatively olivine-poor sediments are not derived from adjacent olivine-rich materials, and therefore, chemical weathering cannot be inferred based on the olivine-thermal inertia relationship alone. We identify two areas (Terra Cimmeria and Argyre Planitia) with significant olivine abundance and thermal inertias consistent with sand, but no adjacent rocky (parent) units having even greater olivine abundances. More broadly, global analysis with TES reveals that the most typical olivine abundance on Mars is ~5-7% and that olivine-bearing (5-25%) materials have a wide range of thermal inertias, commonly 25-600 J m-2 K-1 s-1/2. TES also indicates that the majority of olivine-rich (>25%) materials have apparent thermal inertias less than 400 J m-2 K-1 s-1/2. In summary, we find that the relationship between thermal inertia and olivine abundance alone cannot be used in infer olivine weathering in the examined areas, that olivine-bearing materials have a large range of thermal intertias, and therefore that a complex relationship between olivine abundance and thermal inertia exists on Mars.