Dr. Robyn Waller and Dr. Russell Waller

Abstract: In Waller & Waller (2021), assembled bias names how composing local pieces into a global picture can introduce emergent error—often worsened by discretization, where models must assemble meaning from many chunks. Transformers embrace discretization but pair (discrete) tokens with attention, which mixes representations by similarity (softmax of dot products), letting global structure influence every position. The result is hologram-style robustness: gist permeates across tokens and layers.

We make this link concrete. For single-head softmax attention with the exponential kernel, we write the operator as a normalized interference of two positive plates (Plates): exact in the infinite-feature limit and unbiased at practical widths. This factorization enables simple “scratch tests” and three measurable holographic signatures inside the mechanism:
(1) Blur — baseline-subtracted PSF FWHM that tends to grow under fair channel thinning;
(2) Invariance — row-shape/shift stability under mask swaps;
(3) Uncertainty — attention entropy / KL divergence.

Plates inherit the efficiency of random-feature approaches (e.g., Performer) while adding engineerable, continuous “knobs” that enable in-mechanism detokenization: when gist is stable, runs of near-redundant token steps can be collapsed into short, structured state updates, reducing reliance on external tokenization and supporting continuous inputs. Conceptually, this lets the model use gist and detail simultaneously, qualifying the universality of assembled-bias concerns.

Bios: Robyn Waller is an Associate Professor of Philosophy at the University of Sussex. She is a member of the Sussex Centre for Consciousness Science. She holds a BA in Psychology (Grinnell College), MA in Philosophy of Psychology (King’s College London), and a PhD in Philosophy (Florida State University). She has previously worked in posts across philosophy and the sciences as a Lecturer/Assistant Professor at University of Alabama, King’s College London, Franklin & Marshall College, and Iona University. Her research focuses on philosophy of mind and cognitive science. Her current research concerns the comparative nature of human and AI world models and how neurotechnology impacts what it means to be a human agent.

Russell Waller is an AI specialist with a background in pure mathematics (BA, Mathematics; MSc, PhD, Topology). He is a former CRO of a tech start up, deploying topological data analysis and machine learning for stakeholders. Beyond academic appointments, he has worked directly in industry across a variety of fields, including finance, telecommunications, energy, biomarkers and behavior, and Internet of Things (IoT) to translate abstract mathematical concepts into high-impact solutions.

Talk delivered as part of Sussex AI seminar series on 29th October 2025. You can watch the recording on our YouTube.