- Author
- Lars Niedermeier
- See also
- Chapter 5: Synaptic Plasticity
-
Chapter 7: Monitoring
-
Chapter 9: MATLAB Offline Analysis Toolbox (OAT)
This tutorial shows how to use CARLsim for dopamine-modulated STDP (DA-STDP). While DA-STDP itself was implemented in former versions already, CARLsim 6 now allows to monitor the DA levels and releases with the Offline Analysis Toolkit (OAT).
This tutorials covers:
- Define a network with domaminergic neurons
- Configure DA modulated STDP
- Monitor DA releases and level in the target neuron group
At the end of the tutorial, you will have:
- Built a simple CARLsim program with DA-STDP and simulated re-enforcement learning
- Applied the GroupMonitor to monitor DA levels and visualized the DA releases in Matlab
This tutorial assumes you have covered the tutorials
9.1 Network setup
The following code snippets gives an overview how to setup the network. For further details, please refer to the full source code of this tutorial in %%CARLSIM_ROOT_DIR%%/doc/source/tutorial/9_dastdp.
9.1.1 CONFIG state
Configure the target neuron group g1 with Izhikevich regular spiking neurons. Dopaminergic neurons in group gda project to the target group g1 and induce DA releases. The network's input gin and thalamatic noise g1noise is simulated by a poisson spiking generator.
Connect the input group gin and target group g1 with plastic synapses for STDP and limit the weights to 20. Use fixed synapses for input connections and dopaminergic projections. Parameterize dopamine-modulated STDP for CUBA mode with the parameters set by default. Configure the network in to run in CUBA mode.
sim->
setSTDP(gin, g1,
true, stdp, alphaPlus, tauPlus, alphaMinus, tauMinus);
Configure the decay over time of DA in the target group with tauDP = 2000.0f.
Set an interactive spike generator on the dopminergic neurons to release DA.
9.1.2 SETUP state
Setup the ConnectionMonitor CM to monitor the weight changes by STDP of the plastic synapses. The SpikeMonitor gda logs the firings of the dopaminergic neurons; spikeMonPre and spikeMonPost are used to determine the spike timing of synapses. The GroupMonitor groupMonPost keeps track of the DA level in the target group g1.
To simulate the network input, create a PoissonRate of 4 Hz and set it as spike rate on gin and g1noise.
9.1.3 RUN state
Run the simulation in 1000 ms steps for an arbitrary duration. The default is 60 seconds and can be changed utilizing the command line interface (see 9.3 CLI (optional)).
for (unsigned t = 0; t < duration; t++) {
spikeMonPost->startRecording();
spikeMonPre->startRecording();
groupMonPost->startRecording();
spikeMonPost->stopRecording();
spikeMonPre->stopRecording();
groupMonPost->stopRecording();
...
For each iteration, the spike timing between pre-synaptic neuron and post-synaptic neuron is to be checked. If the post synaptic neuron has fired after the pre-synaptic neuron, the time difference is positiv and indicates a long term potentation (LTP). Then release DA by calling InteractiveSpikeGenerator::setQuotaAll of iSpikeGen.
...
spikesPre = spikeMonPre->getSpikeVector2D()[0];
spikesPost = spikeMonPost->getSpikeVector2D()[0];
for (int j = 0; j < spikesPre.size(); j++) {
for (int k = 0; k < spikesPost.size(); k++) {
int diff = spikesPost[k] - spikesPre[j];
if (diff > 0 && diff <= 20) {
if (damod) {
iSpikeGen->setQuotaAll(1);
releases++;
}
}
}
}
std::vector< std::vector<float> > weights = CM->
takeSnapshot();
weight = weights[0][0];
Finalize the simulation by deleting the CARLsim instance and print a short summary.
...
}
delete sim;
printf("time: %ds on %s synapse: %s da-stdp: %s da-releases: %d weight: %f\n",
duration, mode?"GPU":"CPU", coba?"COBA":"CUBA", damod?"y":"n", releases, weight);
9.2 Results
Build and install the tutorial by activating the cMake option CARLSIM_TUTORIALS.
To run the program on Windows, open a command prompt, change the working directory, and start the executeable with the default parameters. Adapt the procedure to other platforms like Linux as usual.
cd <CMAKE_INSTALL_PREFIX>\tutorials\9_dastdp
dastdp.exe
- See also
- 9.3 CLI (optional)
9.2.1 Terminal output
By default, the logging is set to LoggerMode USER. The standard output begins with the configuration parameters of the network, like CUBA mode, weight change, and STDP parameters. Then the monitors are listed in the initialization section. After the standard summary of CARLsim, a custom printf shows the amount of DA releaeses and the synaptic weight at the end of the simulation.
.*********************************************************************************
.******************** Welcome to
CARLsim 6.0 ***************************
.*********************************************************************************
.***************************** Configuring Network ********************************
Random number seed: 42
E-
STDP enabled
for pre-ex(1) to post-ex(0)
Running
CUBA mode (all synaptic conductances disabled)
Update weight and weight change every 100 ms
Weight Change Decay is enabled
STDP scale factor = 0.050, wtChangeDecay = 0.950
.***************** Initializing GPU 0 Runtime *************************
.******************** Simulation Summary ***************************
Network Parameters: numNeurons = 503 (numNExcReg:numNInhReg = 0.2:0.0)
numSynapses = 502
maxDelay = 1
Random Seed: 42
Timing: Model Simulation Time = 60 sec
Actual Execution Time = 4.30 sec
Speed Factor (Model/Real) = 14 x
Average Firing Rate: 2+ms delay = 0.000 Hz
1ms delay = 0.222 Hz
Overall = 0.222 Hz
Overall Spike Count Transferred:
2+ms delay = 0
1ms delay = 0
Overall Spike Count: 2+ms delay = 0
1ms delay = 6692
Total = 6692
.*********************************************************************************
time: 60s on GPU synapse:
CUBA da-stdp: y da-releases: 17 weight: 5.874695
9.2.2 Offline Analysis
Utilize the OAT to analyze the monitored synaptic weights and DA with Matlab. On Windows, open the subdirectory scripts in the FileExplorer and double click on the Matlab script dastdpOAT.m. Adapt the procedure to other platforms like Linux accordingly.
Validate that Matlab has the containing folder of the script as working directory. Select the script and execute the code with F9. The first line initOAT.m initializes the OAT installed relative to that working directory.
% Synaptic weights
CR = ConnectionReader('../results/conn_pre-ex_post-ex.dat');
[allTimestamps, allWeights] = CR.readWeights();
SR = SpikeReader('../results/spk_DA neurons.dat');
spkData = SR.readSpikes(); % binWindowMs
spkV = spkData(:,1);
idx = (spkV>0);
subplot (4, 1, 1:3)
plot(allTimestamps, allWeights, allTimestamps(idx), allWeights(idx), '+');
title('Synaptic weights with DA releases');
ylabel('Synaptic Weight Strength');
% DA level
GR = GroupReader('../results/grp_post-ex.dat');
[timestamps,data] = GR.readData();
subplot (4, 1, 4)
plot(timestamps, data, 'color', 'r');
title('DA level in target group');
ylabel('DA level');
xlabel('Time (ms)');
The ConnectionReader CR reads the weights stored by ConnectionMonitor::takeSnapshot and plots the weight changes over time. The SpikeReader SR reads the firing of the dopaminergic neurons and mark the times of the DA releases with red crosses (+).
The GroupReader GR reads the DA levels that were recorded by the GroupMonitor over time and plots them as red line in a subplot below the weight changes.
Fig. 1. Synaptic weights (blue) with DA releases and levels (red)
9.3 CLI (optional)
To enable the user to evaluate distinct configurations without compiling the code, the program provides a simple Command Line Interface (CLI).
int main(int argc, char* argv[]) {
unsigned duration = 60; auto arg1 = "1. simulation time in s (default = 60s)";
unsigned coba = 0; auto arg2 = "2. synapse: 1=COBA, 0=CUBA (default)";
unsigned damod = 1; auto arg3 = "3. modulation: 1=DA is released by activating gDA (default), 0= no DA release";
unsigned type = 1; auto arg4 = "4. stdp type: 1=DA_MOD (default), 0=STANDARD";
unsigned mode = 1; auto arg5 = "5. backend 1=GPU (default), 0=CPU";
unsigned log = 0; auto arg6 = "6. log 2=SILENT, 1=DEV, 0=USER (default)";
...
if ((argc == 2 && strncmp("-h", argv[1], 2) == 0) || argc > 7 || mode > 1 || coba > 1 || damod > 1 || type > 1 || log > 2) {
printf("usage: [duration [synapse [modulation [stdp [backend [log]]]]]]\n");
printf("Arguments:\n\t%s\n\t%s\n\t%s\n\t%s\n\t%s\n\t%s\n", arg1, arg2, arg3, arg4, arg5, arg6);
return 0;
}
References
Solving the Distal Reward Problem through Linkage of STDP and Dopamine Signaling (Izhikevich, 2007)
1.8.13 
