Social interactions are mediated by recognition systems, meaning that the cognitive abilities or phenotypic diversity that facilitate recognition may be common targets of social selection. Recognition occurs when a receiver compares the phenotypes produced by a sender with a template. Coevolution between sender and receiver traits has been empirically reported in multiple species and sensory modalities, though the dynamics and relative exaggeration of traits from senders versus receivers have received little attention. Here, we present a coevolutionary dynamic model that examines the conditions under which senders and receivers should invest effort in facilitating individual recognition. The model predicts coevolution of sender and receiver traits, with the equilibrium investment dependent on the relative costs of signal production versus cognition. In order for recognition to evolve, initial sender and receiver trait values must be above a threshold, suggesting that recognition requires some degree of pre-existing diversity and cognitive abilities. The analysis of selection gradients demonstrates that the strength of selection on sender signals and receiver cognition is strongest when the trait values are furthest from the optima. The model provides new insights into the expected strength and dynamics of selection during the origin and elaboration of individual recognition, an important feature of social cognition in many taxa. This article is part of the theme issue 'Signal detection theory in recognition systems: from evolving models to experimental tests'.
