Multisensorial interferences in multimodal communication: the role of olfaction from seduction to aversion.
(IPrA Brussels 2023) by Paul Bouissac.
The research on multimodality bears almost exclusively on visual and acoustic modes of communication. The phenomenology of perception has traditionally dismissed the sense of smell as a notoriously elusive, variable, and highly subjective experience that does not lend itself to objective studies. As Barwich and Smith (2022) have shown in a recent paper, the influence of olfaction on cognition and emotion has been greatly underestimated. During the last two decades, considerable advances have been made in the understanding of the way in which organisms process the information coming from volatile airborne molecules not only through the very large number of receptors in the nasal and vomeronasal epithelium (patches of specialized tissues of the nose) but also through receptors that are located in other parts of the body as German biologist Hanns Hatt has discovered (e.g, Massberg and Hatt 2018). In a report published in 2022 in Nature, Liam Drew documents the progress made in the science of olfaction since Linda Buck and Richard Axel received a Nobel prize in 2004 for uncovering the molecular foundations of the sense of smell. A comprehensive report published online in 2022 by ScienceDirect reviews the main recent advances in the understanding of olfactory systems. In general, what we perceive as “smell” is an olfactory sensation of which we become aware because it stands out in our perception landscape.
However, by necessity, all interactions between organisms, and between organisms and material objects have an olfactory component. “Touch” also plays its part because the dermal contiguity with another human body or any artefact necessarily implies some measure of more or less subtle olfactory perceptions since heat causes the dispersion of molecules. We live in an environment of volatile airborne molecules including pheromones, and we have been endowed by natural selection with an astronomical number of receptors that can discriminate many thousands of olfactory variations in spite of humans being among the less sensitive species to olfaction among the vertebrates. Usually, we don’t perceive smells or odors in our home and outside because our brain does not register small changes in the chemical component of our familiar environment. Feeling safely at home implies, among other things, that there is no intrusive smells that would alert us to react. Visitors, though, have a different perception and form an olfactory image of our home intimately blended with its specific idiosyncratic visual and acoustic landscape. During the recent Covid-19 pandemic, those who were experiencing anosmia and anhedonia due to an infection reported anxiety and a sense of alienation in their own home. Feeling at home implies indeed a critical olfactory dimension. The BA theory of communication that was formulated by Shimizu in the 1990s refers to a shared safe place that creates a sense of meaning and nurtures knowledge creativity. BA must, by necessity, include an olfactory quality. BA is an important redefinition of the notion of context by adding interiority and mindfulness to the spatial frame of communication. It can be confidently asserted that, in the absence of pathological conditions, no human interaction ever occurs in an olfactory vacuum.
The purpose of this paper is to call attention to the crucial role of olfaction in both face-to-face and mediated communication with particular attention given to discrepant information with respect to visual and acoustic intended messages. To approach this dimension of multimodality both intuitively and empirically, it is necessary to distinguish the olfactive baseline that defines the apparent “absence of smell” in the usual environment of the interactants — as we noted above, olfaction is indeed a definite indicator of “familiar space” – from the salient olfactive information that may become a part of the environment, or, from the intraspecific or interspecific interactions with other organisms. Such molecular modifications of the atmosphere within which the interactions take place can be either accidental or artifactual. The multimodal marketing of various products sometimes makes use of this “stealth input” that plays on the association of certain smells with social status or other distinctive cultural categories. Conversely, when the perfume industry creates a new scent, it is marketed thanks to the creation of a multimodal message that includes both image and text that allow the perfume to secure its semiotic niche in the cosmetic culture through the power of metonymy. However, whether deliberate or not, the input of olfaction has always to be taken into consideration. Indeed, olfactory interferences can undermine the intents of the interactants. Molecules generated by the ink of printed material or by various technological devices may play a part in the reception of multimodal messages. Body generated pheromones are also credited, albeit controversially, to impact subtly human interactions because the presence of vomeronasal patches specialized in processing pheromones are a part of the human olfactory system, albeit not as prominently as in other mammals as well as insects such as flies and ants.
The hypothesis of this paper is that a felicitous multimodal communication act depends in a significant manner on its olfactive dimension all the more so as the selective detection of molecular environments is the most ancient adaptation of the earliest organisms. Evolution is conservative and the more recent – in evolutionary time — adaptive processing of acoustic and optic information did not cancel the vital detection of olfactory information signaling nutrient, potential mates, and lethal dangers coming from toxic sources and predators. Guillaume Poncelet and Sebastian Shimeld have provided a detailed account of “The evolutionary origins of the vertebrate olfactory system” in their review article published in 2020 in Open Biology, a publication of the Royal Society. Typically labelled “the sixth sense”, smell is the most primitive sense in primate evolution as it is supported by complex structures that were present within the brains of the very first mammals. Actually, it is probably the most fundamental adaptation of all forms of lifeas recent advances in the understanding of the behavior of insects like ants and Drosophila fruit flies have demonstrated the high-resolution discriminatory power of their olfactory systems that is supported by complex neurological structures. For instance, the latest research reports that ants olfactory bulbs are located in their antennae and command instant behavior in response to molecules that signal danger. The relevance of olfactory information for the survival and reproduction of flies has also been recently demonstrated. These robust olfactory systems are grounded in very ancient adaptations that all forms of organic life, including humans, have inherited.
Research on the pervasive role of olfaction shows that decisions can vary with the olfactory context, a phenomenon that is grounded on more fundamental empirical evidence showing, for instance, that “the valence and salience of individual odorants are modulated by an animal’s innate preferences, learned associations, and internal states, as well as by the context of odorant presentation” (op. cit. 2022).
In general, the question of whether olfaction is a determining factor in multimodal communication is amenable to empirical investigation as it is definitely a falsifiable hypothesis. However, pragmatics has studied the differential contributions of visual and acoustic information in multimodal communication almost exclusively from a phenomenological point of view based on impressionistic judgment rather than precise measurements that must be expressed in milliseconds. Olfactory information has generally been considered as being irrelevant because of its phenomenological elusiveness, although it is definitely amenable to high resolution evaluation, given that there are millions of olfactory neurons in the lining of the human nose, each one binding to a specific molecular odorant. It can beconfidently assumed that all human interactions proceed along an olfactory baseline. It can equally confidently be assumed that any changes, either intended or accidental, occurring in this olfactory baseline has an impact on multimodal communication processes. It may reinforce the message by adding redundancies (for example, perfume of rose associated with both the name and an image) or it may undermine the message by introducing discrepancies (skunk smell associated with the same image of a rose). Considering the evolutionary origins of the vital relevance of the olfaction system in humans, it can be hypothesized that, when streams of sensorial information come into discrepant combinations, olfaction may undermine the intended message by overriding the information value of the other components. These considerations can, at least, help design falsifiable hypotheses to investigate the part olfaction plays in multimodal communication. Such empirical inquiries have been tested in cases involving nematode worms and rodents to show that attraction can be transformed into aversion through the control of olfactory inputs. These considerations can lead to formulating a general hypothesis that is conducive to empirical research. More importantly, this can inspire falsifiable hypotheses through, for instance, varying the olfactory component of a multimodal message and assessing the decisional output or comparatively testing the response to multimodal messages conveyed in an olfactory vacuum. This perspective may also help identify the cause of failed multimodal communications as a part of the forensic toolkit. The well-established fact that synesthesia is experienced by some humans can also open a window of empirical opportunity. How the smell and sounds of colors play out in the effectiveness of communication in these individuals? If synesthesia is the result of recessive genes (or genetic flukes) that resurface in some individuals, it can be assumed that it remains a potent latent factor in communication. If the plausible hypothesis concerning the pervasive and determining factor of olfaction is correct, we could assume that, more generally, olfaction interferes positively or negatively not only with basic survival behavior such as fight or flee, but also with higher communicative and cognitive functions.
An experiment that may tempt some daring psychologist would be to create chess games with pieces made of dark and white chocolate. A condition for playing that experimental game would be that players can eat the pieces they take. Assuming that the players are not allergic to chocolate, would the smell and gustatory attraction significantly interfere with the high cognitive performance required for winning a chess game? Naturally, running this experiment with individuals allergic to chocolate would yield equally relevant result. This suggested experimental project is only in part “tongue in cheek”.
I hope that I have convincingly shown at least that olfaction is a necessary variable of all multimodal communication processes and that it can be empirically tested through the elaboration of falsifiable hypotheses. In spite of the legendary elusiveness of scents, controlled investigations of olfactory interferences with other modalities is now a realistic possibility thanks to technologies such as the recently developed “scentography”, an apparatus dubbed “odor camera”. As it was shown in a paper delivered at the 20th ACM International Conference on Multimodal Interaction, in October 2018, the experimental use of smell is now operational (Maggioni et al. 2018). Let us hope that, in the not too distant future, olfaction will become a part of the holistic investigation of the pragmatics of multimodal communication.
Be the first to comment