This article provides evidence for an extension of structural information theory, a theory which describes perceptual organization formally, into a more general theory of representation that takes the role of organizations obtained earlier into account. In eight experiments, subjects study series of colored dots. Each series contained 6-8 dots of different colors, and viewing time was between 400 ms and 60 s. In several experimental procedures, subjects recalled the series presented. The experiments systematically varied the economy of the organizations of the series as described by structural information theory as well as a number of aspects of the context in which the series were presented, like order of presentation of a series, order between the series, viewing span, viewing time, and recall procedure. Recall was influenced both by the economy as well as by the context. It is shown how the context influences the relative strength of the most economical organization as compared with rival, local organizations. A structured network model is presented that accounts for the influence of both the economy and the context.