This is a follow-up on the previous post “Mirroring the Timing of Neural Processes”, to set up a few more issues that lie ahead of the brain time theory.
We defined Brain time theories as theories that subscribe to the following principles:
The thesis of minimal delay: Our experiences are only ever delayed by the time it takes for the stimulus to reach us and for our neural mechanism to process it.
The thesis of temporal isomorphism: The subjective time of experience is determined by the time of the neural process realising the experience
Together these entail the following view: The time at which sensory features are processed in the brain determines the time at which they are represented, and no extra delays are added to make up for external or internal latency differences.
External latency can be the varying amount of time it takes for sound vs. light to reach the boundary surface of sensory mechanisms. Internal latencies differences describe differences in the processing of for example colour vs. motion.
Since much behaviour requires us to interact with changes that occur at the timescale of tens of milliseconds, perceptions do need to be rapidly available to guide our actions. For example, speech recognition requires extremely fast processing to segregate phonemes from each other, if phonetic information is not neatly segregated and rapidly available it would render speech recognition impossible. This is neatly explained by the brain time theory, where sensory information is available ASAP.
At other times, however, we need perceptions that integrate a lot of sensory information that takes place across different modalities and timescales. This is required to make sense of how we can perceive an event and how it unfolds in time in as coherent integrated perceptions.
A big problem in explaining this latter kind of perception that is integrated across modalities and timescales is the problem of desynchronization.
Desynchronization describes the issue of how the different kinds of sensory information of a specific event become desynchronized as they are processed by our various neural mechanisms. They do so because of:
(i) Our neural mechanisms responsible for processing sensory information differ concerning the modality and timescale of the stimuli to be processed. These different processes take varying amounts of time, and:
(ii) The different stimuli received when experiencing some event constantly change. But the relative time at which specific stimuli of some event change do not match up with the relative time at which these specific stimuli are processed.
Due to (i) and (ii) we should expect the time at which different stimuli of some external events change and the time at which these stimuli are finished processing to become desynchronised. This phenomenon is also described by Hinze Hogendoorn (2022, 129).
“[I]nformation from a single sensory event becomes available to perception not at any single moment, but over a range of moments. For the same reason, the most recent information available to perception at any given moment will have originated from different moments in the environment for different features, and therefore do not belong together to the same percept. This causes a temporal binding problem: as the brain processes the continuous stream of desynchronised sensory input, how does it infer what happened when?”
One must explain how this temporal binding is achieved to account for perceptions that are temporally bound across multiple modalities and timescales
According to the principle of temporal isomorphism, perceptions are available the moment they are processed. The principle of minimal neural delay then makes sure that perceptions realised by different neural mechanisms, do not wait for each other to be presented together in coherent perceptions.
But if the time at which perceptions are processed and thus experienced differs from sensory mechanism to sensory mechanism. Then the relative time at which different kinds of sensory information (e.g., colour and motion) are experienced does not match up with the relative time at which the stimuli (colour and motion) in the external events occur.
The sensory information of our perceptions thus becomes desynchronised with the external world that we are supposed to keep track of. So, how can we, as Hogendoorn writes, “infer what happened when”, given that desynchronisation takes place?
If we accept the principle of isomorphism, the principle of minimal delay. and the fact that our sensory processing systems become desynchronised. We end up with a system, that would make deducing the actual time of external events, that are of a multi-modal nature, practically impossible.
But this is not impossible! We are very temporally accurate and can engage with a rapidly changing environment as if there was almost no delay between changes that take place in event time and how they are perceived in subjective time.
The brain-time theory has with its two principles created a very rigid perceptual system that does not allow for any recalibration or added neural latency compensations to explain how desynchronised sensory information is integrated into coherent percepts. The problem of desynchronisation thus challenges the inflexibility of brain time theories.
They can adapt by either rejecting the principle of temporal isomorphism thus allowing for the storage of sensory information that is not instantly represented but integrated across multiple modalities and timescales before a percept is created.
Or brain time theories can reject the principle of minimal delay and argue that the sensory processing of some mechanisms is delayed by the amount of time it would take for the information of other mechanisms to be processed so that they can be represented together. The latter option would be compatible with the principle of temporal isomorphism if one held that the neural process realising an experience lies at the moment where the relevant sensory information is integrated.
Either way, it seems one or more of the two principles must go if brain time theories are to deal with the problem of desynchronisation.