In the construction of neural networks involving associative recall, information is sometimes best encoded with a local representation. Moreover, a priori knowledge can lead to a natural selection of connection weights for these networks. With predetermined and fixed weights, standard learning algorithms that work by altering connection strengths are unable to train such networks. To address this problem, this paper derives a supervised learning rule based on gradient descent, where connection weights are fixed and a network is trained by changing the activation rule. It incorporates both traditional and competitive activation mechanisms, the latter being an efficient method for instilling competition in a network. The learning rule has been implemented, and the results from several test networks demonstrate that it works effectively.