Previous studies have demonstrated that the early retinotopic cortex (ERC, i.e., V1/V2/V3) is highly associated with the lateral occipital complex (LOC) during visual perception. However, it remains largely unclear how to evaluate their associations in quantitative way. The present study tried to apply a multivariate pattern analysis (MVPA) to quantify the neural activity in ERC and its association with that of the LOC when participants saw visual images. To this end, we assessed whether low-level visual features (Gabor features) could predict the neural activity in the ERC and LOC according to a voxel-based encoding model (VBEM), and then quantified the association of the neural activity between these regions by using an analogical VBEM. We found that the Gabor features remarkably predicted the activity of the ERC (e.g., the predicted accuracy was 52.5% for a participant) instead of that of the LOC (4.2%). Moreover, the MVPA approach can also be used to establish corresponding relationships between the activity patterns in the LOC and those in the ERC (64.2%). In particular, we found that the integration of the Gabor features and LOC visual information could dramatically improve the 'prediction' of ERC activity (88.3%). Overall, the present study provides new evidences for the possibility of quantifying the association of the neural activity between the regions of ERC and LOC. This approach will help to provide further insights into the neural substrates of the visual processing.