We present a new method for the estimation of the next brain MR scan in a longitudinal tumor follow-up study. Our method effectively incorporates information of the past scans in the time series to predict the future scan of the patient. Its advantages are that it requires no user intervention and does not assume any particular tumor growth model. Instead, the patient-specific tumor growth parameters are estimated individually from the past patient scans. To validate our method, we conducted an experimental study on four patients with Optic Path Gliomas (OPGs) and four patients with glioblastomas multiforma (GBM), each scanned at five time points. The tumor volumes in the predicted and actual future scans, both segmented by an expert radiologist, yield a mean volume overlap difference of 13.65% for the OPG patients, and 34.23% for the GBM patients.