The human microbiota may influence the effectiveness of drug therapy by activating or inactivating the pharmacological properties of drugs. Computational methods have demonstrated their ability to screen reliable microbe-drug associations and uncover the mechanism by which drugs exert their functions. However, the previous prediction methods failed to completely exploit the neighborhood topologies of the microbe and drug entities and the diverse correlations between the microbe-drug entity pair and the other entities. In addition, they ignored the case that a microbe prefers to associate with its own specific drugs. A novel prediction method, PCMDA, was proposed by learning the neighborhood topologies of entities, inferring the association preferences, and integrating the features of each entity pair based on multiple biological premises. First, a knowledge graph consisting of microbe, disease, and drug entities is established to help the subsequent integration of the topological structure of entities and the similarity, interaction, and association relationship between any two entities. We generate various topological embeddings for each microbe (or drug) entity through random walks with neighborhood restarts on the microbe-disease-drug knowledge graph. Distance-level attention is designed to adaptively fuse neighborhood topologies covering multiple ranges. Second, the topological embeddings of entities imply the latent topological relationships between entities, while the relational embeddings of entities are derived from the semantics of connections among the entities. The topological structure and relational semantics of entities are fused by a designed knowledge graph learning module based on multilayer perceptron networks. Third, considering the preference that each microbe tends to especially associate with a group of drugs, information-level attention is designed to integrate the dependency between microbial preference and the candidate drug. Finally, a dual-gated network is established to encode the features of a microbe-drug entity pair from multiple biological perspectives. The comparative experiments with seven state-of-the-art methods demonstrate PCMDA's superior performance for microbe-drug association prediction. The case studies on three drugs and the recall rate evaluation for the top-ranked candidates indicate that PCMDA has the capability of discovering reliable candidate microbes associated with a drug. The datasets and source codes are freely available at https://github.com/pingxuan-hlju/pcmda.