Firing-Rate Models

Firing-rate models simplify neuron dynamics by considering firing-rate as the main representation of information processing in the brain. Firing-rate models look at units, groups of neurons whose mean activity is similar

Recurrent feedback is input to a unit that depends on the unit’s own activity. Of course, units also impact each other.

Firing-rate models are simpler and therefore faster. This is great because it gives them more explanatory power in describing network behavior. However, in reality changes on a small timescale do affect how a system reacts.

Mean-field theory ignores correlations between variables and smooths out the effect of fluctuations

Firing-rate models are mean-field theories since they only represent mean firing rate.

These models will use different increasing functions to represent a neuron’s firing rate according to a given input. For example, sigmoid ($\frac{1}{1+e^{-x}}$), power-law function, threshold linear response, etc…

Simulation requires solving a set of coupled ODE.

1. The inputs to the units determine their firing rates
2. The firing rates determine the new inputs

${\tau }_{r_i}\frac{d{r}_i}{dt}=-r_i+f_i(\left\{W_{ji}{s}_j\right\})$
${\tau }_{s_i}\frac{d{s}_i}{dt}=-s_i+F(r_i)$

• $W_{ji}$ denotes the strength of connection from unit $j$ to unit $i$
• $s_i$ denotes the fraction of downstream synaptic channels open
  • Here we will simplify $f_i(\left\{W_{ji}{s}_j\right\})=f_i\left(\sum _j{W}_{ji}{s}_j\right)=f_i(S_i)$
  • $S_i$ simplify represents the total synaptic input
• $F(r_i)$ is the synaptic gating function, and is usually taken as a linearly increasing function of the presynaptic firing rate. If the firing rate has some maximum value, then it is easy to write $F(r_i)=\frac{r_i}{r_i^{(\text{max})}}$

If ${\tau }_{s_i}\ll {\tau }_{r_i}$ then one can treat it as updating instantly and only solve one set of equations,
${\tau }_{r_i}\frac{d{r}_i}{dt}=-r_i+f_i\left(\sum _j{W}_{ji}\frac{r_j}{r_j^{(\text{max})}}\right)$

Note that Dale’s principle is not relevant with regards to units (because why would it)

A bistable unit can maintain spiking activity at two distinct firing rates when receiving the same level of input (typically zero).

1. The lower firing rate is either quiescence or a low firing rate corresponding to spontaneous neural activity.
2. The higher firing rate is initiated by an excitatory stimulus and persists afterwards
   This kind of unit acts as a memory circuit; its activity is history-dependent. These have been recorded in monkeys during short-term memory tasks. An antagonist of NMDA prevents such responses, suggesting their important in short-term memory maintenance.

Bistability requires a process of positive feedback to lead away from one equilibrium and a process of negative feedback to settle at a new equilibrium

A supralinear function’s gradient increases along the x-axis, also called a convex function

There are several firing rate limiting processes,

1. Firing rate saturation; real neurons have hard physical limits on firing rates
2. The number of available receptors limits the firing rate, so neurotransmitters with slower time-constants can stabilize activity
3. The number of release-ready vesicles also limits the input to postsynaptic cells.

Fixed points are points of the system in which all variables stop changing. There are stable and unstable fixed points, which depends on the rate and feedback curves.

The synaptic time constant ${\tau }_s$ limits the mean synaptic response $⟨s⟩$ due to a presynaptic neuron firing spikes as a Poisson process at rate $r$ via
$⟨s⟩=\frac{\alpha p_{r}r{\tau }_r}{1+\alpha p_{r}r{\tau }_s}$ where $p_r$ is the release probability of each docked vesicle per spike and $\alpha$ is the maximum fraction of postsynaptic receptors bound by neurotransmitters when all vesicles are released

In a simple model $s=F(r)$. If the dynamics of $s$ are simulated then a more complex equation is,
$\frac{ds}{dt}=-\frac{s}{{\tau }_s}+\alpha p_{r}r(1-s)$

The effective time constant for changes in $s$ following changes in $r$ becomes ${\tau }_{eff}(r)=\frac{{\tau }_s}{1+\alpha p_{r}r{\tau }_s}$

In the simplest feedback curve of a bistable system, the feedback and firing rate curves intersect at three points, two of which are stable. A line attractor, on the other hand, is a continuous range of values in which a system is stable.

These line attractors can act as integrators and effectively sum up inputs. Any input moves the system along its line where it stays, an effect called parametric memory.

However, it is not entirely clear if neural circuits ever really produce line attractors.

Decision-Making Circuits

Models of decision-making address three distinct issues,

1. Accounting for experimental data, such as the proportion and time taken for different responses, depending on a stimulus
2. Ascertaining which models produce more optimal behavior
3. Accounting for biological constraints in the makeup of neural circuits

Integration of evidence is the optimal way of accumulating information from a stimulus

The Wang model accounts for the activity observed with regards to decisions between two alternatives. It shows how two groups of neurons corresponding to the two alternatives could both reproduce the behavior and the ramping activity of the “winning” group of neurons

Winner-takes-all circuits have multiple distinct units within which only one unit can sustain high activity, because of cross-inhibition

During perceptual decisions, neural activity appears to ramp up gradually over a longer time period (closer to a second). This requires greater fine-tuning in a circuit comprised of neurons with time constants on the order of a few milliseconds

$\tau \frac{d{r}_1}{dt}=-r_1+W_s{r}_1+W_x{r}_2+i_1$
$\tau \frac{d{r}_2}{dt}=-r_2+W_s{r}_2+W_x{r}_1+i_2$
$\tau \frac{d{r}_d}{dt}=-r_d+(W_s-W_x)r_d+i_d$
$\frac{d{r}_d}{dt}=-\frac{r_d}{{\tau }_{eff}}+\frac{i_d}{\tau }$

Perfect integration of evidence is when $W_s=W_x=1$ and ${\tau }_{eff}\to \infty$

Forced response paradigm is a study in which the time of a response is fixed, as opposed to a free response paradigm

The relative cost of an incorrect response versus a slower correct response determines whether faster decisions with more errors are optimal or not. If the goal is to maximize reward rate, then longer intervals between trials favors slower, more accurate responses

In general, there is a speed-accuracy tradeoff, where faster responses are less accurate

An alternative to the gradual ramping of neural activity arising from integration is the possibility of abrupt state-transitions between different stable states. This would appear similar in measurements that average over many samples

Bias is a tendency to prefer one alternative over others before a stimulus is presented. If prior information indicates that one choice is more likely to be the correct one, then we should bias our choice according to such a prior by Bayes’ Theorem. The same effect should happen if one outcome is more rewarding.

In a model, one can achieve bias by applying constant input to the favored alternative. The circuit could also be initialized in a state closer to one threshold.

Excitatory and Inhibitory Feedback

Even on the larger scale of firing rate modeling, neural systems reveal oscillatory tendencies. Oscillatory fields are created when neural activity is synchronized among many neurons aligned in a similar direction.

Oscillations vary with our mental state and are task dependent. The gamma rhythm is in the range of 30-80 hertz and is associated with increased attention and memory processing in its localized area.

A subset of models for gamma oscillations relies on the coupling between excitatory neurons (pyramidal cells) and inhibitory neurons (fast-spiking interneurons). These PING models rely on self-excitation and feedback inhibition

In the PING model, oscillations happen from a repeating cycle of quick self-excitation, followed by decaying inhibition.

Other models use only synchronized inhibition to create oscillations. These models rely on either outside excitation or high spontaneous neural activity.

Power spectrum is a plot of oscillatory power versus frequency to indicate the dominant frequencies in any time-dependent signal

The Fourier transform is a way to separate a time-dependent function into its oscillating components

Orientation Selectivity

The primary visual cortex or V1 is the region at the back of the brain which receives most direct visual input from retina via the thalamus

The lateral geniculate nucleus is a region of the thalamus receiving inputs from the optic nerve

Orientation selectivity is the preference of many neurons in V1 for edges or gratings oriented at a particular angle

One interesting feature is the way neurons respond to higher contrast. They are contrast invariant, which is to say the amplitude of their tuning curve increases but does not broaden as contrast increases. This means V1 neurons cannot only receive input from the excitatory thalamic neurons, since then you would expect the tuning curve to broaden.

A couple of models could be used to represent this feature,

1. Excitatory cells receiving inhibition from opposite orientations. If inhibition scales with contrast in the same way as excitation scales with contrast then the tuning curve would seem to stay the same shape
2. Another model uses recurrent feedback in the excitatory unit along with an inhibitory unit

These models imply a ring-like connectivity structure, so that neurons of perpendicular selectivity can inhibit each other.

A ring attractor is a network in which neurons can be labelled by a variable with circular symmetry, such as orientation, and thus can be arranged into a ring

If the recurrent excitatory feedback is increased then activity can remain after stimulus offset, acting as a short term memory

A circuit can have three classes of stable activity

1. Only inactive state, can produce contrast-invariant responses during stimulus presentation
2. Active and inactive state, can provide memory for the location of a stimulus
3. Only active state, can provide information that always exists (like body position)

Simulations suggest that angular integration is more robust in a ring model with inhibitory feedback connections than with excitatory feedback connections