Fluid displacement within layered porous media is more complex than in nonlayered ones. Most of the previous studies placed a focus on the porous media with layerings perpendicular to the flow direction, and the effects of pore topology were often ignored. Therefore, this study aims to reveal the flow physics in porous media with layering parallel to the flow direction by accounting for the specific pore topology. Based on the phase-field method, a series of pore-scale numerical simulations were performed to investigate the dynamic displacement processes within layered porous media under various capillary numbers and wettability. The results showed that oil recovery was strongly affected by the heterogeneity of porous media in the capillary fingering regime compared to the viscous one. Besides, the alternative advancing phenomenon of the water fingers was suppressed in layered porous media. Compared with water wetting conditions, the viscous fingering regime of the invading water is more pronounced under oil wetting conditions. Last but not least, the center of mass of water flow paths can be used for fingering regime identification. These findings contribute to our better understanding of the flow dynamics of fluid displacement within layered porous media, which are of great significance to the industry related to oil recovery, underground storage of carbon dioxide and hydrogen gas, etc.