Featured pre-print

Modelling Phenomenological Differences in Aetiologically Distinct Visual Hallucinations Using Deep Neural Networks

Keisuke Suzuki, Anil K. Seth, David J. Schwartzman

Summary of full article

Visual hallucinations (VHs) are perceptions of objects or events in the absence of the sensory stimulation that would normally support such perceptions. VHs offer fascinating insights into the mechanisms underlying perceptual experience, yet relatively little work has focused on understanding the differences in the phenomenology of VHs associated with different aetiologies (causes). For instance, VHs arising from neurological conditions, visual loss, or psychedelic compounds have substantial phenomenological differences between them.

Here, we examine the potential mechanistic basis of these differences by leveraging recent advances in visualising the learned representations of a coupled classifier and generative deep neural network. Using this coupled deep neural network architecture, we generated synthetic VHs that captured three dimensions of hallucinatory phenomenology which broadly characterise variations in VHs: their realism (veridicality), dependence on sensory input (spontaneity), and complexity.

We verified the validity of this approach experimentally in two separate studies that investigated variations in hallucinatory experience in neurological and CBS patients and people with recent psychedelic experience. Both studies first verified that the three phenomenological dimensions usefully distinguished the different types of hallucination, and then asked whether the appropriate synthetic VHs were able to capture specific aspects of hallucinatory phenomenology for each aetiology. In both studies, we found that the relevant synthetic VHs were rated as being most representative of each group’s hallucinatory experience, compared to other synthetic VHs produced by the model.

Our results highlight the phenomenological diversity of VHs associated with distinct causal factors and demonstrate how a neural network model of visual phenomenology can successfully capture the distinctive visual characteristics of hallucinatory experience. The novel combination of deep neural network architectures and a computational neurophenomenological approach provides a powerful approach towards closing the loop between hallucinatory experiences and their underlying neurocomputational mechanisms.