Unifying the Fragments of Time Perception
Can we get experience and information processing to fit?
Today I want to present a new way to set out the problem that has kept me interested in the philosophy and psychology of time perception. To do so I largely rely on Philosopher Gerardo Viera’s paper “The Perceived Unity of Time” from 2020.
Viera argues that we experience time as unified. By this, he means that the world seems to consist of a single unified timeline on which all events are experienced to occur within. When we think about how we perceive and experience events we seem to place them all along a single temporal dimension. Every memory we entertain and every event we experience seem to take place within the same timeline.
But this description of the perceived unity of time is not matched when we look at the brain. The different mechanisms that underpin time perception, let us call these timekeeping mechanisms, do not have a unified structure as do our perceptions. Rather, the range of timekeeping mechanisms our brain employs is abundant and they work in independent and different ways. These timekeeping mechanisms are fragmented to such a degree that it becomes hard to explain how the perceived unity of time of our experience is constructed.
So on one hand we have the perceived unity of time and on the other hand, we have a lot of fragmented timekeeping mechanisms. This becomes a problem once we want an explanation for how the mechanisms of the brain can be responsible for the experiences we enjoy. If we cannot give such an explanation it becomes mysterious both why our temporal experiences are the way they are and why the timekeeping mechanisms of the brain are the way they are. Why is there this apparent gap between these two aspects of time perception that otherwise should just be two sides of the same story: Our lived experience and the mechanisms that underpin those experiences?
Let us unpack these two aspects in terms of their experiential and behavioural functions:
The perceived unity of time
According to Gerardo Viera, the perceived unity of time can be set out in terms of two perceptual capacities. Viera dubs these two capacities: localisation and comparability.
Localisation can be understood as how events seem to be located in time relative to a specific anchored moment, the moment we live in at every waking moment. Some events seem to have occurred in the past, and some have yet to occur, but they all occur in the past or the future relative to this moment.
The perceptual capacity localisation is also behaviourally important. To interact properly with our environment we must be able to predict and represent information about when things occur or are about to occur. More importantly, we must represent or predict information about when things occur or are about to occur in relation to the moment we live in.
Failing to match behaviour with an accurate estimation of external temporal relations results in people failing to catch balls flying through the air, in cats jumping too late to capture their prey, and in sports it causes sports judges to call the wrong sports verdicts (as you can read I know my sports).
Knowing when some event will occur only allows for coordination of behaviour if you also know when that event will occur relative to your ‘position’ in time.
Comparability (the other perceptual capacity), can be described as the way we seem to be able to effortlessly compare temporal properties across modalities and timescales. How we compare temporal properties picked up from auditory and temporal properties from visual cues is easy and requires no real translation.
Temporal perceptual experiences present temporal properties of different modalities to be of the same psychological phenomenon. There is no distinct ear-time that is experientially different from eye-time.
Compare this to how the experience of different modalities (sight, smell, touch) seems to differ in the spatial case. Even though we have only one visual field, there are clear differences between sensing tactile features in space and sensing visual features in space.
Consider comparing the intensity of a tactile sensation to the intensity of a light. Such comparisons might be possible to an extent, but as Viera mentions there is a “lingering awkwardness” to them, that completely disappears in the temporal case.
Behaviourally cross-modal temporal representations often seem to be integrated, and become part of the same perception, without us even noticing any such integration taking place. An effect called temporal ventriloquism reveals this kind of behaviour.
In cases of temporal ventriloquism, two sensory features are presented in sequence, a noise followed by a visual flash, the ventriloquist effect kicks in when the judgment of when these two stimuli occur is shifted toward the more reliably localisable feature. In other words, one feature ‘binds’ the other, thus bringing them closer together in time.
The temporal representations from different modalities interact in ways that seem to support the idea that we behaviourally treat these as representing information about the same type of property in the world, a temporal property. These experiments reveal how we behaviourally seem to compare temporal information from different modalities, without noticing perceptual discrepancies about how these two bits of temporal information are represented in the world.
What about the other aspect of time perception, the timekeeping mechanisms of the brain?
The fragmentation of timekeeping mechanisms
I am not going into the technical details of how our timekeeping mechanisms are fragmented, although I have written on this, we can take a look at these details in another post. Here I’ll just discuss what it means for such mechanisms to be fragmented and provide a list of ways in which they probably are fragmented.
Timekeeping mechanisms are whatever array of mechanisms of the brain that fixes the temporal contents of our representations. When I say that these mechanisms are fragmented I mean that they differ with regards to their representational goals, content, and ways of representing and that there is no agreement about a neat taxonomical or functional specific way to set out these mechanisms.
One can investigate whether an array of mechanisms supposed to underpin the same larger goal, allowing for the perception of time, are fragmented by testing whether the different mechanisms can dissociate. That is whether their functions can be selectively intervened upon in ways that leave the other mechanisms unaffected by the intervention.
I hold that there are at least five dimensions along which timekeeping mechanisms can dissociate in this way. These are the dimensions of 1. timescales, 2. modalities, 3. sensory feature type, 4. temporal property types, and 5. temporal judgments.
By timescales, I mean that there are different timekeeping mechanisms in play for dealing with phenomena occurring at different timescales. For example mechanisms for perceiving phenomena at a timescale of 100ms differs from those used for phenomena occurring at a timescale of 1 second.
Timekeeping mechanisms also work differently for different modalities, these different mechanism can to some extent be affected individually.
The sensory feature type, light, colour, motion, flicker, etc. also seem to affect timekeeping mechanisms differently. The temporal properties of an object changing motion are for example not the same as those for the colour change of that same object, the temporal properties of these feature types can dissociate.
The temporal property types of duration, sequences, and rates can also be selectively manipulated in a variety of ways.
Last but not least there is also a large range of evidence that shows how temporal judgment tasks greatly influence our timekeeping capacities, which can be taken as evidence for the fact that our temporal perceptions are goal-/task-dependent and non-unitary across different kinds of temporal judgment.
The problem summarized
In the sections above we have described two very different aspects of time perception that seem to be in tension. From the experiential or subjective perspective, time seems to be perceptually unified in the ways described by the perceptual capacities of localisation and comparability.
This stands in sharp contrast to the perspective we get on time perception when we look at the information-processing story of the mechanisms that underpin timekeeping. These timekeeping mechanisms are far from unified and there is no reason why we should expect a single timeline to automatically emerge from this range of mechanisms. There is also nothing that indicates why the temporal information represented by these fragmented mechanisms should be comparable.
It would almost be more likely that the representational content of these mechanisms would be incommensurable as our different capacities seem to be directed at different representational purposes.
If we accept that both the accounts of our experience of time and the information processing underlying our time-keeping mechanisms are true, how then do we account for both of these aspects of time perception within a single unified theory? Maybe there is no hope for a unified theory of time perception. Still, if this is so we would be left with a problematic gap between our perceived unity of time and our perceptual system that is supposed to give rise to these perceptions of unity.
I find this problem intriguing, and I find no currently existing solution to be persuasive, we might look at some of these proposals in the next article. Some day in the future I might also reveal my take on a solution, but it is still too controversial and too vaguely defined to be allowed to see the light of day.