Accurate calculation of the binding affinity of small molecules to proteins has the potential to become an important tool in rational drug design. In this study, we use the free energy perturbation (FEP) method with restraints to calculate the standard binding free energy of five ligands (ACPA, AMPA, CNQX, DNQX, and glutamate) to the glutamate receptor GluA2, which plays an essential role in synaptic transmission. To deal with the convergence problem in FEP calculations with charged ligands, we use a protocol where the ligand is coupled in the binding site while it is decoupled in bulk solution simultaneously. The contributions from the conformational, rotational, and translational entropies to the standard binding free energy are determined by applying/releasing respective restraints to the ligand in bulk/binding site. We also employ the confine-and-release approach, which helps to resolve convergence problems in FEP calculations. Our results are in good agreement with the experimental values for all five ligands, including the charged ones which are often problematic in FEP calculations. We also analyze the different contributions to the binding free energy of each ligand to GluA2 and discuss the nature of these interactions.