A large variety of severe medical conditions involve alterations in microvascular circulation. Hence, measurements or simulation of circulation and perfusion has considerable clinical value and can be used for diagnostics, evaluation of treatment efficacy, and for surgical planning. However, the accuracy of traditional tracer kinetic one-compartment models is limited due to scale dependency. As a remedy, we propose a scale invariant mathematical framework for simulating whole brain perfusion. The suggested framework is based on a segmentation of anatomical geometry down to imaging voxel resolution. Large vessels in the arterial and venous network are identified from time-of-flight (ToF) and quantitative susceptibility mapping (QSM). Macro-scale flow in the large-vessel-network is accurately modelled using the Hagen-Poiseuille equation, whereas capillary flow is treated as two-compartment porous media flow. Macro-scale flow is coupled with micro-scale flow by a spatially distributing support function in the terminal endings. Perfusion is defined as the transition of fluid from the arterial to the venous compartment. We demonstrate a whole brain simulation of tracer propagation on a realistic geometric model of the human brain, where the model comprises distinct areas of grey and white matter, as well as large vessels in the arterial and venous vascular network. Our proposed framework is an accurate and viable alternative to traditional compartment models, with high relevance for simulation of brain perfusion and also for restoration of field parameters in clinical brain perfusion applications.