The interaction of small Amyloid-β (Aβ) oligomers with the lipid membrane is an important component of the pathomechanism of Alzheimer's disease (AD). However, oligomers are heterogeneous in size. How each type of oligomer incorporates into the membrane, and how that relates to their toxicity, is unknown. Here, we employ a single molecule technique called Q-SLIP (Quencher-induced Step Length Increase in Photobleaching) to measure the membrane insertion of each monomeric unit of individual oligomers of Aβ42, Aβ40, and Aβ40-F19-Cyclohexyl alanine (Aβ40-F19Cha), and correlate it with their toxicity. We observe that the N-terminus of Aβ42 inserts close to the center of the bilayer, the less toxic Aβ40 inserts to a shallower depth, and the least toxic Aβ40-F19Cha has no specific distribution. This oligomer-specific map provides a mechanistic representation of membrane-mediated Aβ toxicity and should be a valuable tool for AD research.