Perineuronal nets (PNNs) are a specialized extracellular matrix that surround certain populations of neurons, including (inhibitory) parvalbumin (PV) expressing-interneurons throughout the brain and (excitatory) CA2 pyramidal neurons in hippocampus. PNNs are thought to regulate synaptic plasticity by stabilizing synapses and as such, could regulate learning and memory. Most often, PNN functions are queried using enzymatic degradation with chondroitinase, but that approach does not differentiate PNNs on CA2 neurons from those on adjacent PV cells. To disentangle the specific roles of PNNs on CA2 pyramidal cells and PV neurons in behavior, we generated conditional knockout mouse strains with the primary protein component of PNNs, aggrecan (Acan), deleted from either CA2 pyramidal cells (Amigo2 Acan KO) or from PV cells (PV Acan KO). Male and female animals of each strain were tested for social, fear, and spatial memory, as well as for reversal learning. We found that Amigo2 Acan KO animals, but not PV Acan KO animals, had impaired social memory and reversal learning. PV Acan KOs, but not Amigo2 Acan KOs had impaired contextual fear memory. These findings demonstrate independent roles for PNNs on each cell type in regulating hippocampal-dependent memory. We further investigated a potential mechanism of impaired social memory in the Amigo2 Acan KO animals and found reduced input to CA2 from the supramammillary nucleus (SuM), which signals social novelty. Additionally, Amigo2 Acan KOs lacked a social novelty-related local field potential response, suggesting that CA2 PNNs may coordinate functional SuM connections and associated physiological responses to social novelty.