Spike timing neural network model of conscious visual perception

—The aim of the paper is to investigate the inﬂuence of thalamo-cortical connectivity on the conscious perception of visual stimuli. We conducted simulation experiments changing the key parameters of our spike timing neural network model of visual perception and decision making that are supposed to be related to conscious perception, namely bottom-up and top-down connections between thalamic relay, including Thalamic reticulate nucleus (TRN) and Lateral geniculate nucleus (LGN), and primary visual cortex (V1). The model output, that is perceptual based decision in the lateral intrapare-ital (LIP) area of the brain for left or right saccade generation, was observed. Conclusions about the inﬂuence of altered key parameters on the ability of our model to take proper decision were commented in respect to the observed activity in the brain areas responsible for conscious visual perception and decision making.


I. INTRODUCTION
Since the earliest days of psycho-physiology, there has been a debate about the link between sensation, perception, attention, and consciousness. The main question is: what happens to a sensory signal in the brain when it reaches a conscious stage of processing as opposed to being processed outside of awareness? In search of an answer to this question the concept of "Neural correlates of consciousness" is introduced that represents the set of neuronal events and mechanisms generating a specific conscious perception. Based on it in [3], [4] consciousness is viewed as a state-dependent property of some complex, adaptive, and highly interconnected biological structures in the brain. A model of consciousness is a theoretical description that relates brain phenomena such as fast irregular electrical activity and widespread brain activation to expressions of consciousness such as qualia [19].
Recent studies on neural correlates of consciousness are a continuation of the research initiated at the end of the 19th century [12]. The contemporary trend in this intensively developing nowadays area of research was initiated in the 1990s by the development of an empirical approach focusing on visual awareness because the visual system was already intensively investigated [2], [3], [4], [10]. Since then, consciousness research became more diverse but its link to visual perception continued [17], [18]. Irrespective of the intense interest and research efforts in studying the consciousness, there is still no consensus about the neural correlates of consciousness, i.e. what are the minimal neural mechanisms that are jointly sufficient for any one conscious perception, thought or memory, under constant background conditions [3]. It is still unclear which brain regions are essential for conscious experience.
Recently, a dominant trend is to view consciousness as emerging from interactions between distributed networks of neurons and especially from the global activity patterns of cortico-cortical and thalamo-cortical loops [5], [6], [7], [14], [21], [22], [23]. In [15] a hypothesis was proposed that the thalamus is the primary candidate for the location of consciousness since it has been referred to as the gateway of nearly all sensory inputs to the corresponding cortical areas. This theory was supported by numerous works. Lately, a view of thalamocortical processing is proposed in [20] where two types of thalamic relays are defined: first-order relays receiving subcortical driver input, e.g. retinal input to the lateral geniculate nucleus, and higherorder relays, receiving driver input from layer 5 of the cortex, that participate in cortico-thalamocortical circuits. Recent findings [8] support the important role of the lateral geniculate nucleus in the emergence of consciousness and provide a more complex view of its connections to the other parts of the thalamus and the visual cortex. In [1] it was suggested that the hallmark of conscious processing is the flexible integration of bottomup and top-down thalamo-cortical streams and a novel neurobiology theory of consciousness called Dendritic Information Theory, was proposed.
We have already developed a hierarchical spike timing model of visual information perception and decision making including the detailed structure of the thalamic relay composed by laretal geniculate nucleus, thalamic reticulate nucleus and interneurons [11]. The model was implemented in NEST Simulator [9] on the supercomputer Avitohol.
Here we investigate the influence of the thalamo-cortical connectivity on the conscious perception of visual stimuli by changing the key parameters of our model that are supposed to be related to conscious perception, namely the bottom-up and top-down connections between the thalamic relay and primary visual cortex (V1). The model output -perceptual based decision for left or right saccade generation -was observed. Conclusions about the influence of the altered key parameters on the ability of our model to take proper decision were commented in respect to the observed activity in the areas V1, Middle temporal (MT), Medial superior temporal (MST) and LIP areas.
The structure of the rest of the paper is as follows: next section presents briefly our model architecture and its parameters under investigation; the simulation results are presented and discussed in sections III and IV; the paper finishes with the concluding remarks pointing out the directions of our further work.

II. MODEL STRUCTURE
The details of the structure of our spike timing hierarchical model of visual information processing and perceptual decision making with reinforcement learning were reported in [11]. For the aim of the current simulation investigations we use only its perceptual-based decision part shown on Fig.  1.
Each colored rectangle on Fig. 1 represents a group of neurons positioned on a regular twodimensional grid. Each group, called further layer, corresponds to a brain structure involved in visual information perception as follows: retinal ganglion cells (RGC); Lateral geniculate nucleus (LGN); . Each layer has a special role in the visual information processing as follows: RGC in eyes transform the light to electrical signal fed into the brain via the optic nerve; LGN, TRN and IN has a role of relay structure forwarding the information to the visual cortex; V1 detects orientation of the visual stimulus; further MT detects the direction of movements while MST has more complicated role to detect the patterns of movement (here expansion/contraction from/to a given focal point); finally LIP collects the processed visual information and takes a perceptual based decision (in this case left or right expansion center of the stimulus).
The arrows denote the connections between layers, called synapses. The details of connectivity, described in our previous works [11], are based on the literature information. Briefly, each neuron has its own receptive field -area of neurons from a given layer that it is connected to -depending on the function of the layer it belongs to as well as on its position within its layer.
The dimensions of all model building blocks are shown in Table I  The connections of interest in current investigation are those between the thalamus (including LGN, TRN and IN layers) and the visual cortex (V1 layer). The V1 neurons have orientation sensitivity due to their elongated receptive fields defined by Gabor functions. Their orientation and phase parameters were determined so as to achieve the typical for the mammalian brain "pinwheelstructure" (for more information see [16]). They are separated into four groups -two excitatory (E1 and E2) and two inhibitory (I1 and I2) -connected via lateral connections based on their distance and roles as in [13]. Fig. 2 shows the designed in this way strengths of forward connections (from the thalamus to V1).
The feedback connections from V1 to the thalamus are proportional to the feedforward once.
In the current work we scale the feedforward and feedback connections to investigate their role in the conscious visual information perception.

III. SIMULATION EXPERIMENTS
The model input stimulus consisted of moving dots expanding from a left focal point as shown on Fig. 3 (for more details see our previous works [11], [16]). The visual stimulation lasted  for 1670 milliseconds. First 150 milliseconds were "washed out" since they were needed for RGC spatio-temporal filters to accumulate the visual information at the beginning of the stimulation.
We conducted simulation experiments by varying the scaling factors of both feedforward and feedback connections between the thalamus and V1. Table II summarizes the experiments. Sign + denotes the cases with successful propagation of the visual information from RGC up to LIP area while those marked with − are the cases without transmission of the visual information to V1.
The smallest values of the feedforward connections (scaled by 0.1 and below) do not allow propagation of the visual information to the primary visual cortex thus preventing the perceptual based decision while the minimal feedback connectivity does not have such deterioration effect.
In order to investigate in details the effect of varying feedforward/feedback connectivity we  Table II) for all considered model layers (V1, MT, MSTe and LIP left respectively).

Figures 4-7 show the simulation results for the experiments with successful propagation of the visual information (marked by + in
The top parts of the figures 4-7 show firing rates during stimulation in the observed layers of the model. In order to distinguish clearly differences in spiking activity, bottom parts of the figures 4-7 show the mean and the variance of the above spiking frequencies. As it was expected, the biggest differences in spiking activity caused by scaling of the feedback/feedforward connectivity were observed in V1 since it is the first structure influenced directly by the thalamus. Fig. 4 shows that with the increase of both feedforward and feedback connections the initiation of spiking activity, e.g. first reaction to the visual stimulus, begins earlier. In case of smallest feedforward connectivity without feedback connections (feedback scaling 0.0 and feedforward scaling 0.5) it begins at about 107th millisecond of stimulation, while in the last case (feedback/feedforward scaling 5.0) it begins at about 61st millisecond. The increased feedforward connectivity also led to the increase in spiking activity by the end of stimulation.
The spiking activity in the next (MT) layer was influenced in similar manner like that in V1 while the MSTe and LIP areas show almost similar activity for all experimental cases.
While the mean of the V1 firing rate increases with both scaling parameters (Fig. 4), its variance shows lowest values in cases of middle (2.0) and lowest (0.5) feedforward scaling coefficients.
The mean spiking frequency in the MT area shows a bit different tendency: the biggest mean frequency was observed in the case of the biggest feedforward scaling coefficient in combination with lowest feedback scaling coefficient. The variance of spiking activity in MT has also irregular dependence on the thalamus-V1 connectivity showing bigger values not only in the case with the biggest scaling parameters but also in the cases with middle feedforward and low feedback scaling as well as with low feedforward and high feedback scaling coefficients (Fig. 5).
The observed in the MT area effect spreads also to the MSTe and LIP areas (figures 6 and 7). While the biggest mean frequencies in the MSTe area were observed for the biggest feedforward and lowest feedback scaling coefficients, its biggest variance was in the case of middle feedforward scaling (2.0) in combination with low feedback scaling (0.01) coefficients. The strongest mean firing activity in the decision area (left LIP area in our example) moves further to the lowest feedforward scaling (0.5) in combination with middle (2.0) feedback scaling while the biggest variance in LIP spiking activity was observed for relatively low feedback/feedforward connectivity with scaling coefficients 0.5 and 1.0 respectively.
In summary, the biggest mean firing rates move through the model areas from the case of biggest feedforward/feedback scaling coefficients in V1 to the case of lowest feedforward and middle feedback scaling coefficients in the LIP area. This result shows the significance of feedback connectivity from V1 to the thalamus. In all cases even small feedback connectivity increases the mean spiking activity through all the model areas in comparison with cases of missing feedback thalamo-cortical connectivity. However, after some threshold the higher feedback scaling coefficients have suppressing effect on the spiking activity in the MT, MST and LIP areas.

V. CONCLUSION
The presented in this paper simulation experiments with varying feedforward/feedback thalamo-cortical connectivity demonstrated the significance of the feedback from the visual cortex to the thalamus. Our results revealed that the relation between the two way connectivity strengths and the spiking activity in the visual information processing areas MT, MST and LIP following the visual cortex V1 is not straightforward and should be investigated in deeper details.
We have to account that our model connectivity was designed based on the literature information and needs further experimental clarification. The dimension of the test model was minimal so its scaling up to more realistic number of neurons will make the simulation experiments more realistic too.
Our planes for future work include exploitation of the experimental data (MRI and fMRI) revealing the real brain connectivity in order to make our model more realistic. We expect to obtain such a data by cooperation with the partners from COST action "Neural architectures of consciousness" in the near future. The implementation of the model on the supercomputer Avitohol will allow a significant increase of its dimension that is our second aim for the future work.
Acknowledgments: The reported work is supported by the Bulgarian Science Fund grant No KP-06-COST/9 from 2021 "Supercomputer simulation investigation of a neural model of brain structures involved in conscious visual perception" by a co-funding scheme of COST action CA18106 "Neural architectures of consciousness". SN was supported by the Bulgarian National Scientific Program Young Scientists and Postdocs, Module Young Scientists.