Learning and Plasticity

Hebbian plasticity

Hebbian plasticity is a change in synaptic strength produced by a positive correlation between presynaptic and postsynaptic spikes

The mechanisms generally depend on an increase in calcium concentration within the postsynaptic cell in the vicinity of the synapse, either through calcium channels, NMDA receptors, or internal stores

A presynaptic spikes causes glutamate to be released onto the postsynaptic receptors for a short amount of time; then, if the postsynaptic cell’s membrane potential rises, the NMDA channels (glutamate receptors) open and admit calcium ions

Long-term depression is necessary to counteract the long-term potentiation that results from Hebbian plasticity. Therefore, active cells ought to weaken their excitatory connections to less active excitatory cells. Even with this change, the system would seem to tend towards an extreme where feedback causes a subset of synapses to be strengthened to their biological maximum

Hebbian plasticity can be simulated in a simple way with firing rate models, where the connection strength between two units changes according to the covariation in their firing rates

$\tau \frac{d{W}_{ij}}{dt}=\Theta (r_i-r_r)\cdot \Theta (r_j-r_r)$ where $\Theta (x)$ is the Heaviside function (1 if $x$ is positive, 0 otherwise) and $r_r$ is the threshold rate for inducing plasticity

One could also use a continuous rate of change function, like $\tau \frac{d{W}_{ij}}{dt}=r_i{r}_j(r_j-r_r)$ (positive is they are the same sign, negative otherwise)

These networks can demonstrate,

1. Pattern completion, producing a correct pattern based on prior examples when provided with a partial or corrupted input
2. Pattern separation, producing proper responses for different patterns, depending on which pattern the input more closely resembles

Spike-Timing Dependent Plasticity

STDP means changes in connection strength that depend on the relative timing of spikes, not only rates

The rules listed so far reflect the shortening version of Hebb’s postulate, “cells that fire together wire together.” In reality, causality and timing are important

In some experiments, strengthening of the synaptic connection in one direction corresponded with weakening in the other direction

• In addition, a minimum number rate of pairs is required
• A minimum and maximum number of pairings will produce significant change
• A high amount of variability in synaptic change is observed

While these factors will be ignored for simplification, a few other factors are necessary to consider

1. Whether to only consider successive pairs of spikes are all potential pairs with units (matters depending on spike rate)
2. Whether to use a batch updating method or a continuous updating method

A standard model of STDP looks like,
$\Delta W_{ij}=-A_{-}\cdot e^{-(t_i-t_j)/{\tau }_{-}}$ with $t_i>t_j$
$\Delta W_{ij}=-A_{+}\cdot e^{-(t_j-t_i)/{\tau }_{+}}$ with $t_j>t_i$

This can be updated at each spike or after a batch, but for the continuous method it is good to continually update the variables,
$\frac{d{a}_i}{dt}=-\frac{a_i}{{\tau }_{+}}$ (total presynaptic impact)
$\frac{d{e}_i}{dt}=-\frac{e_i}{{\tau }_{-}}$ (total postsynaptic impact)

Then at each postsynaptic spike $j$, the strength is incremented by $\Delta W_{ij}=A_{+}{a}_i$ and at each presynaptic spike $i$, the strength is decremented by $\Delta W_{ij}=A_{-}{e}_j$

The area between each curve and the $x$-axis is given by $A_{+}{\tau }_{+}$ and $A_{-}{\tau }_{-}$. If spikes are completely uncorrelated then potentiation is proportional to the former and depression is proportional to the latter. In this scenario, it’s necessary that $A_{-}{\tau }_{-}>A_{+}{\tau }_{+}$, so that

1. Increasing a subset of synapses strengthens their postsynaptic activity
2. Increased postsynaptic activity leads to more pairs of spikes and shorter time intervals, leading to enhanced plasticity
3. If other presynaptic neurons have uncorrelated spikes then their input synapses become depressed
   Overall, this creates competition, where increase in one subset decreases another

STDP can enable sequence learning, where neurons learn to fire in a particular order

To better model STDP interactions, more complicated protocols exist, like triplet STDP

Note that empirical rules for altering synaptic strength are just approximations, ignoring many intracellular processes. For many neurons, coming up with empirical rules is more efficient and simpler

• For example, sometimes calcium spikes are necessary for inducing plasticity
• Since calcium spikes are correlated with high postsynaptic activity, a rule can be made simply requiring a high rate of postsynaptic spikes for plasticity

NMDA receptors on calcium channels are an important mechanism in enabling STDP. NMDA receptors on a postsynaptic cell require the binding of glutamate (released by presynaptic cells) and depolarization (caused by postsynaptic spiking). The time constant of depolarization is much shorter than that of NMDA receptors, so causality is enforced

Membrane potential is usually also an important factor in models of synaptic plasticity

• Depression arises when a presynaptic spike follows a period of depolarization
• Potentiation arises at a high postsynaptic membrane potential if it follows a presynaptic spike

If another variable would be simulated, then calcium is the best choice. Since its concentration typically varies a lot, each synapse requires a separate variable. Somatic calcium concentration can be used for cell-wide homeostatic regulation

Homeostasis

Negative feedback is very important in maintaining a system. Neural circuits have their own needs for regulation, such as the right level of firing

Neural circuit behavior depends sensitively on parameters like synaptic connection or different conductances. These variables are also maintained over time through the production and processing of proteins

Neurons must also maintain the right level of sensitivity to inputs. One way to achieve this is if a neuron has a mechanism to monitor its firing rate and adjust the strengths of synapses or excitability accordingly

This has been observed to happen; if neurons are prevented from spiking with tetrodotoxin for a day or more, they become more excitable, whereas applications of bicuculline lead to a reduction in excitatory synaptic strengths

A calcium-dependent negative feedback pathway could support homeostasis; Increasing firing rate leads to an increase in somatic calcium, reducing the conductance of sodium channels and decreasing the neuron’s firing rate

These mechanisms can be simulated in a simple way by decreasing excitability according to average firing rate,
${\tau }_g\frac{d{G}_E}{dt}=r_{goal}-r(t)$ or ${\tau }_{th}\frac{d{V}_{th}}{dt}=r_{goal}-r(t)$ with extremely long time scales

Homeostatic feedback is likely necessary to ensure neurons retain their role as they change in size, since the rate of protein transcription would seem to change at a rate non-proportional to surface area

Supervised Learning

Supervised learning is learning that depends on feedback based on the outcome of a behavior

While unsupervised learning allows our brains to extract common features from the environment

Reinforcement learning is supervised learning where the feedback signal is a scalar quantity; A better than expected outcome produces a positive reinforcement signal while a worse outcome produces a negative signal

Reinforcement learning feedback has no direction, it only rewards or punishes so is easiest to look at in situations with discrete alternatives

Neuroeconomics is the study of the neural basis of decision making in situations where the outcomes of those decisions are quantifiable

Unconditioned stimulus cause a response without any need for training

Conditioned stimulus cause a response after training

Classical conditioning is associating a conditioned stimulus with an unconditioned stimulus

Operant conditioning (also called instrumental conditioning) is the altering of an animal’s behavior such that some responses are rewarded and some are punished. This is more relevant to neural circuits

Dopamine plays a key role in conditioning and generally signals a reward prediction error

Reinforcement learning can help an animal transition from exploring possibilities to exploiting the best actions

In the weather prediction task, a subject must learn how well different stimuli predict a response. A three-component plasticity rule is sufficient to produce the correct behavior.

Presynaptic Rate	Postsynaptic Rate	Prediction Error	Change in Strength
High	High	+	+
High	Low	+	-
High	High	-	-
High	Low	-	+

This makes sense, however optimal decision-making requires accounting for the probability of the response given the stimulus, and this does not do so. For a rarer stimulus, this makes a difference

Eyeblink conditioning is an interesting experiment where an animal is conditioned to blink after hearing a tone in order to avoid a gust of air

The protocol could be one of delay conditioning if the tone remains on or trace conditioning in which the tone leaves a pause (this difference is important because trace conditioning requires the hippocampus for short-term memory)