├── .gitignore ├── README.md ├── SectionB └── RW_Softmax.mlx ├── SectionC ├── FLUX Workshop Section C.pptx ├── MLE_fmincon.m └── log_lik.m ├── SectionD ├── .DS_Store ├── Flux_GroupD_combined.pdf ├── Flux_GroupD_combined.pptx ├── model_recovery │ ├── .DS_Store │ ├── README.md │ ├── fit_simulated_data.m │ ├── lik_funs │ │ ├── .DS_Store │ │ ├── decay_lik.m │ │ ├── null_lik.m │ │ ├── one_LR_lik.m │ │ └── two_LR_lik.m │ ├── model_recovery.m │ ├── models │ │ ├── .DS_Store │ │ ├── sim_decay.m │ │ ├── sim_null.m │ │ ├── sim_oneLR.m │ │ └── sim_twoLR.m │ ├── parameter_recovery.m │ ├── simulate_choice_data.m │ └── simulated_data │ │ ├── .DS_Store │ │ ├── 200_trials_data_10-Sep-2021.mat │ │ ├── 200_trials_fits_10-Sep-2021.mat │ │ ├── 20_trials_data_10-Sep-2021.mat │ │ ├── 20_trials_fits_10-Sep-2021.mat │ │ ├── data_10-Sep-2021.mat │ │ ├── data_27-Aug-2021.mat │ │ ├── data_28-Aug-2021.mat │ │ ├── data_30-Aug-2021.mat │ │ ├── fits_10-Sep-2021.mat │ │ ├── fits_27-Aug-2021.mat │ │ ├── fits_28-Aug-2021.mat │ │ └── fits_30-Aug-2021.mat └── parameter_recovery │ ├── Palminteri_Wyart_Koechlin_Trends_2017.pdf │ ├── alpha0.5_100trials.png │ ├── alpha0.5_300trials.png │ ├── decay_learningrate.png │ ├── model_falsification.m │ ├── model_falsification.mlx │ ├── model_falsification.png │ ├── parameter_recovery_simulate.m │ ├── parameter_recovery_simulate.mlx │ └── sampleFromArbitraryP.m ├── TutorialMaterials ├── README.md └── Tutorial1 │ ├── README.md │ ├── estimate_QnLL_gridsearch.m │ ├── fullrecovery_QnLL_fmincon.m │ ├── simulateQ_one.m │ ├── simulateQ_sess.m │ └── simulateQ_sub.m ├── archive └── Example_Live_Script.mlx ├── general_functions ├── learn_RescWagn.m └── softmax.m └── slides_morning ├── 2021_09_17_Welcome_FLUX_workshop.pdf ├── FLUXMorningSession_Introduction.pdf └── FLUXSectionB_How to develop a computational model.pptx.pdf /.gitignore: -------------------------------------------------------------------------------- 1 | *.asv 2 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 2021 Flux Computational Modelling in Development Workshop 2 | 3 | This repository contains slides, code, and further information on all the sessions that took place in this workshop. 4 | 5 | 6 | Programme: 7 |
time (GMT)
13:00Welcome from the Organisers (Ali Cohen & Tobias Hauser)
Computational modelling in development: Past, current, and future directions (Cate Hartley)
13:30What is Computational Modelling? Introduction and examples
a. What is a computational model and why do we use it? (Nadescha Trudel & Alisa Loosen)
b. How to develop a computational model? (Tricia Seow, Sam Hewitt, & Noam Goldway)
c. Principles of modelling and model fitting (Magda Dubois, Naiti Bhatt, Greer Bizzell-Hatcher, & Vasilisa Skvortsova) 
d. Model comparison, selection & validation (Kate Nussenbaum, Johanna Habicht, & Vasilisa Skvortsova) 
16:00Break
17:00Parallel modelling tutorials:
a. Inferring cognitive models of reinforcement learning from choice data (Maël Lebreton & Stefano Palminteri)
b. Computational modeling of goal-directed and habitual reinforcement-learning strategies (Claire Smid & Wouter Kool)
c. Computational models of human gaze data (Angela Radulescu)
d. Uncovering heterogeneity in preferences and behavior with finite mixture models (Adrian Bruhin)
e. An introduction to drift diffusion modeling (Wenjia Joyce Zhao & Ian Krajbich)
19:00Panel discussion: Promises and Pitfalls in Developmental Computational Modelling
19:30virtual drinks / find-a-modeler & find-an-experimentalist session
8 | 9 | 10 | 11 | Recorded videos can be re-watched here
12 | Computational Modelling: https://vimeo.com/608444451/9d4affa0c3
13 | Tutorial 1: https://vimeo.com/608403671/51a07abb82
14 | Tutorial 2: https://vimeo.com/608475995/65f3ea0c24
15 | Tutorial 4: https://vimeo.com/608496420/85baf1c378
16 | Tutorial 5: https://vimeo.com/608416142/0343e7cac3
17 | Panel Discussion: https://vimeo.com/608510255/48c7e34b41
18 | 19 | 20 | 21 | Content was primarily created by the Hartley (https://www.hartleylab.org/) and Hauser (www.devcompsy.org) Labs 22 | -------------------------------------------------------------------------------- /SectionB/RW_Softmax.mlx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionB/RW_Softmax.mlx -------------------------------------------------------------------------------- /SectionC/FLUX Workshop Section C.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionC/FLUX Workshop Section C.pptx -------------------------------------------------------------------------------- /SectionC/MLE_fmincon.m: -------------------------------------------------------------------------------- 1 | clear all 2 | n=2; 3 | thetatrue=[1 1 2 2 0.2]; 4 | mu = [0 0]; %mean and variance covariance matrix 5 | sd = [1 thetatrue(2*n+1); thetatrue(2*n+1) 1]; 6 | data = randi([0 8],100,4); % make integer array of 100 observations of 4 variables 7 | X=data(:,1:n); % x values are first n columns of array 8 | Y=data(:,n+1:size(data,2)); % y values are rest of columns 9 | B1(:,2)=-X(:,1); 10 | B1(:,1)=-1; 11 | B2(:,2)=-X(:,2); 12 | B2(:,1)=-1; 13 | C1=(all(bsxfun(@eq,Y,[0 0]),2)); % change to 0s and 1s using eq function and return logical true if all values in second dimension are nonzero/true 14 | C2=1-C1; % return 0 or 1, depending on previous result 15 | cdf=@(x) mvncdf( [B1*[x(1);x(3)], B2*[x(2);x(4)] ] ,mu,[1 x(5); x(5) 1]); 16 | options=optimset('Algorithm',... 17 | 'interior-point','Display','iter','MaxIter',10000,'TolX',10^-30,'TolFun',10^-30); 18 | theta0=thetatrue; % set initial theta randomly 19 | [theta,fval,exitflag,output]=... 20 | fmincon(@(x) log_lik(x,cdf,C1,C2),theta0,[],[],[],[],[-Inf; -Inf; -Inf; -Inf; -1], ... 21 | [+Inf; +Inf; +Inf; +Inf; 1],[],options); -------------------------------------------------------------------------------- /SectionC/log_lik.m: -------------------------------------------------------------------------------- 1 | function val = log_lik(theta,cdf,C1,C2) 2 | %LOG_LIK Calculate negative log likelihood 3 | % Inputs: theta, cdf, weights 4 | g=cdf(theta); 5 | val=-sum(C1.*log(g)+C2.*log(1-g)); 6 | end 7 | 8 | -------------------------------------------------------------------------------- /SectionD/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/.DS_Store -------------------------------------------------------------------------------- /SectionD/Flux_GroupD_combined.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/Flux_GroupD_combined.pdf -------------------------------------------------------------------------------- /SectionD/Flux_GroupD_combined.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/Flux_GroupD_combined.pptx -------------------------------------------------------------------------------- /SectionD/model_recovery/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/.DS_Store -------------------------------------------------------------------------------- /SectionD/model_recovery/README.md: -------------------------------------------------------------------------------- 1 | # Model recoverability scripts for Flux 2021 Computational Modeling Workshop 2 | 3 | To generate simulated choice data: 4 | simulate_choice_data.m 5 | 6 | To fit models to data: 7 | fit_simulated_data.m 8 | 9 | To plot parameter recoverability: 10 | parameter_recovery.m 11 | 12 | To plot model recoverability: 13 | model_recovery.m 14 | 15 | Note: The parameter and model recoverability scripts assume that the models used to simulated the choice data and the models used to fit the data were 1.) the same and 2.) run in the same order. 16 | 17 | Questions? Email katenuss@nyu.edu. 18 | -------------------------------------------------------------------------------- /SectionD/model_recovery/fit_simulated_data.m: -------------------------------------------------------------------------------- 1 | %% Fit models to simulated choice data %% 2 | % Kate Nussenbaum - katenuss@nyu.edu 3 | % Flux Computational Modeling Workshop 4 | % Last edited: 8/30/21 5 | 6 | % clear everything 7 | clear; 8 | 9 | %load path to likelihood functions 10 | addpath('lik_funs'); 11 | 12 | %% VARIABLES TO MODIFY %% 13 | %Data file name 14 | save_filename = ['simulated_data/200_trials_fits_', date]; 15 | 16 | % Data to load 17 | load('simulated_data/200_trials_data_10-Sep-2021'); 18 | data = sim_data; 19 | 20 | % How many iterations to run per participant 21 | niter = 5; 22 | 23 | % Models to fit 24 | models = {'1LR', 'decay', 'null'}; 25 | 26 | %determine the number of subjects 27 | n_subjects = length(data(1).sub_data); 28 | 29 | %% FIT MODELS TO DATA %% 30 | 31 | %---------------------------------------------% 32 | % Loop through models, datasets, and subjects % 33 | %---------------------------------------------% 34 | for d = 1:size(sim_data, 2) 35 | 36 | %get data for this dataset 37 | dataset_data = data(d).sub_data; 38 | 39 | %print message about which dataset is being fit 40 | fprintf('Fitting dataset %d out of %d...\n', d, size(sim_data, 2)) 41 | 42 | for m = 1:length(models) 43 | model_to_fit = models{m}; 44 | 45 | %print message about which subject is being fit 46 | fprintf('Fitting model %d out of %d...\n', m, length(models)) 47 | 48 | % model specific info 49 | if strcmp(model_to_fit, '1LR') 50 | n_params = 2; %tau, alpha 51 | lb = [.1, 1e-6]; %lower bounds of the parameters 52 | ub = [3, 1]; %upper bounds of the parameters 53 | function_name = 'one_LR_lik'; 54 | elseif strcmp(model_to_fit, '2LR') 55 | n_params = 3; %tau, alpha_pos, alpha_neg 56 | lb = [.1, 1e-6, 1e-6]; 57 | ub = [3, 1, 1]; 58 | function_name = 'two_LR_lik'; 59 | elseif strcmp(model_to_fit, 'decay') 60 | n_params = 3; %tau, alpha_init, eta 61 | lb = [.1, 1e-6, 1e-6]; 62 | ub = [3, 1, 1]; 63 | function_name = 'decay_lik'; 64 | elseif strcmp(model_to_fit, 'null') 65 | n_params = 0; %no parameters 66 | function_name = 'null_lik'; 67 | end 68 | 69 | % convert function name to function 70 | fh = str2func(function_name); 71 | 72 | % generate matrices to save data 73 | [logpost, negloglik, AIC, BIC] = deal(nan(n_subjects, 1)); 74 | [params] = nan(n_subjects, n_params); 75 | 76 | 77 | %loop through subjects 78 | parfor s = 1:n_subjects 79 | sub_data = dataset_data(s); 80 | 81 | fprintf('Fitting subject %d out of %d...\n', s, n_subjects) %print message saying which subject is being fit 82 | 83 | %if null model, compute negative log likelihood directly 84 | if strcmp(model_to_fit, 'null') 85 | logpost(s) = NaN; 86 | params(s, :) = NaN; 87 | negloglik(s) = -1*sum(log(.5*ones(length(sub_data.choices), 1))); 88 | AIC(s) = 2*negloglik(s); 89 | BIC(s) = 2*negloglik(s); 90 | else %otherwise fit models 91 | for iter = 1:niter % run niter times from random initial conditions, to get best fit 92 | % choosing a random number between the lower and upper bounds 93 | % (defined above) to initialize each of the parameters 94 | starting_points = rand(1,length(lb)).* (ub - lb) + lb; % random initialization 95 | 96 | % Run fmincon (unless null model) 97 | [res,nlp] = ... 98 | fmincon(@(x) fh(sub_data.blocks, sub_data.choices, sub_data.rewards, x),... 99 | starting_points,[],[],[],[],lb, ub,[],... 100 | optimset('maxfunevals',10000,'maxiter',2000, 'Display', 'off')); 101 | 102 | %flip sign to get log posterior (if priors are in models, if no priors, this will be the log likelihood) 103 | logp = -1 * nlp; 104 | 105 | %store results if minimum is found 106 | if iter == 1 || logpost(s) < logp 107 | logpost(s) = logp; 108 | params(s, :) = res; 109 | negloglik(s) = fh(sub_data.blocks, sub_data.choices, sub_data.rewards, res); %fit model w/ 'wining' parameters to get the negative log likelihood 110 | AIC(s) = 2*negloglik(s) + 2*length(res); 111 | BIC(s) = 2*negloglik(s) + length(res)*log(length(sub_data.choices)); 112 | end 113 | 114 | end 115 | end 116 | end 117 | 118 | results.logpost = logpost; 119 | results.params = params; 120 | results.negloglik = negloglik; 121 | results.AIC = AIC; 122 | results.BIC = BIC; 123 | 124 | %save for each model 125 | model_fits(d, m).results = results; 126 | model_fits(d, m).fit_model = model_to_fit; 127 | model_fits(d, m).sim_model = data(d).function; 128 | 129 | end 130 | end 131 | 132 | 133 | %----------------------------% 134 | % Save model-fitting results % 135 | %----------------------------% 136 | save(save_filename, 'model_fits'); 137 | 138 | 139 | 140 | -------------------------------------------------------------------------------- /SectionD/model_recovery/lik_funs/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/lik_funs/.DS_Store -------------------------------------------------------------------------------- /SectionD/model_recovery/lik_funs/decay_lik.m: -------------------------------------------------------------------------------- 1 | %% KN 2 | % Likelihood function for one LR model 3 | 4 | function [lik] = decay_lik(blocks, choices, rewards, x) 5 | 6 | tau = x(1); 7 | alpha_init = x(2); 8 | eta = x(3); 9 | 10 | % Initialize log likelihood at 0 11 | lik = 0; 12 | 13 | % Loop through trials 14 | for trial = 1:length(choices) 15 | 16 | % At the start of each block, initialize value estimates 17 | if trial == 1 || blocks(trial) ~= blocks(trial-1) 18 | val_ests = [.5 .5]; 19 | end 20 | 21 | % Determine choice probabilities 22 | ev = exp(val_ests./tau); 23 | sev = sum(ev); 24 | choice_probs = ev/sev; 25 | 26 | % Determine the choice the participant actually made on this trial 27 | trial_choice = choices(trial); 28 | 29 | %Determine the probability that the participant made the choice they 30 | %made 31 | lik_choice = choice_probs(trial_choice); 32 | 33 | %update log likelihood 34 | lik = lik + log(lik_choice); 35 | 36 | % Get the reward the participant received on this trial 37 | trial_reward = rewards(trial); 38 | 39 | %Compute prediction error 40 | PE = trial_reward - val_ests(trial_choice); 41 | 42 | %determine alpha 43 | if trial == 1 44 | alpha = alpha_init; 45 | else 46 | alpha = alpha_init/(1+eta*(trial-1)); 47 | end 48 | 49 | % Update value estimates for next trial 50 | val_ests(trial_choice) = val_ests(trial_choice) + alpha * PE; 51 | 52 | end 53 | 54 | % Put priors on parameters 55 | %lik= lik+log(pdf('beta', alpha, 1.1,1.1)); 56 | %lik= lik+log(pdf('gam', beta, 4.82, 0.88)); 57 | 58 | %flip sign of loglikelihood (which is negative, and we want it to be as close to 0 as possible) so we can enter it into fmincon, which searches for minimum, rather than maximum values 59 | lik = -lik; 60 | 61 | -------------------------------------------------------------------------------- /SectionD/model_recovery/lik_funs/null_lik.m: -------------------------------------------------------------------------------- 1 | %% KN 2 | % Likelihood function for one LR model 3 | 4 | function [lik] = null_lik(blocks, choices, rewards, x) 5 | 6 | % Initialize log likelihood at 0 7 | lik = 0; 8 | 9 | % Loop through trials 10 | for trial = 1:length(choices) 11 | 12 | choice_probs = [.5 .5]; 13 | 14 | % Determine the choice the participant actually made on this trial 15 | trial_choice = choices(trial); 16 | 17 | %Determine the probability that the participant made the choice they 18 | %made 19 | lik_choice = choice_probs(trial_choice); 20 | 21 | %update log likelihood 22 | lik = lik + log(lik_choice); 23 | 24 | 25 | end 26 | 27 | % Put priors on parameters 28 | %lik= lik+log(pdf('beta', alpha, 1.1,1.1)); 29 | %lik= lik+log(pdf('gam', beta, 4.82, 0.88)); 30 | 31 | %flip sign of loglikelihood (which is negative, and we want it to be as close to 0 as possible) so we can enter it into fmincon, which searches for minimum, rather than maximum values 32 | lik = -lik; 33 | 34 | -------------------------------------------------------------------------------- /SectionD/model_recovery/lik_funs/one_LR_lik.m: -------------------------------------------------------------------------------- 1 | %% KN 2 | % Likelihood function for one LR model 3 | 4 | function [lik] = one_LR_lik(blocks, choices, rewards, x) 5 | 6 | tau = x(1); 7 | alpha = x(2); 8 | 9 | % Initialize log likelihood at 0 10 | lik = 0; 11 | 12 | % Loop through trials 13 | for trial = 1:length(choices) 14 | 15 | % At the start of each block, initialize value estimates 16 | if trial == 1 || blocks(trial) ~= blocks(trial-1) 17 | val_ests = [.5 .5]; 18 | end 19 | 20 | % Determine choice probabilities 21 | ev = exp(val_ests./tau); 22 | sev = sum(ev); 23 | choice_probs = ev/sev; 24 | 25 | % Determine the choice the participant actually made on this trial 26 | trial_choice = choices(trial); 27 | 28 | %Determine the probability that the participant made the choice they 29 | %made 30 | lik_choice = choice_probs(trial_choice); 31 | 32 | %update log likelihood 33 | lik = lik + log(lik_choice); 34 | 35 | % Get the reward the participant received on this trial 36 | trial_reward = rewards(trial); 37 | 38 | %Compute prediction error 39 | PE = trial_reward - val_ests(trial_choice); 40 | 41 | % Update value estimates for next trial 42 | val_ests(trial_choice) = val_ests(trial_choice) + alpha * PE; 43 | 44 | end 45 | 46 | % Put priors on parameters 47 | %lik= lik+log(pdf('beta', alpha, 1.1,1.1)); 48 | %lik= lik+log(pdf('gam', beta, 4.82, 0.88)); 49 | 50 | %flip sign of loglikelihood (which is negative, and we want it to be as close to 0 as possible) so we can enter it into fmincon, which searches for minimum, rather than maximum values 51 | lik = -lik; 52 | 53 | -------------------------------------------------------------------------------- /SectionD/model_recovery/lik_funs/two_LR_lik.m: -------------------------------------------------------------------------------- 1 | %% KN 2 | % Likelihood function for two LR model 3 | 4 | function [lik] = two_LR_lik(blocks, choices, rewards, x) 5 | 6 | tau = x(1); 7 | alpha_pos = x(2); 8 | alpha_neg = x(3); 9 | 10 | % Initialize log likelihood at 0 11 | lik = 0; 12 | 13 | 14 | % Loop through trials 15 | for trial = 1:length(choices) 16 | 17 | % At the start of each block, initialize value estimates 18 | if trial == 1 || blocks(trial) ~= blocks(trial-1) 19 | val_ests = [.5 .5]; 20 | end 21 | 22 | 23 | % Determine choice probabilities 24 | ev = exp(val_ests./tau); %multiply inverse temperature * value estimates and exponentiate 25 | sev = sum(ev); 26 | choice_probs = ev/sev; %divide values by sum of all values so the probabilities sum to 1 27 | 28 | % Determine the choice the participant actually made on this trial 29 | trial_choice = choices(trial); 30 | 31 | %Determine the probability that the participant made the choice they 32 | %made 33 | lik_choice = choice_probs(trial_choice); 34 | 35 | %update log likelihood 36 | lik = lik + log(lik_choice); 37 | 38 | % Get the reward the participant received on this trial 39 | trial_reward = rewards(trial); 40 | 41 | %Compute prediction error 42 | PE = trial_reward - val_ests(trial_choice); 43 | 44 | % Update value estimates for next trial 45 | if PE > 0 46 | val_ests(trial_choice) = val_ests(trial_choice) + alpha_pos * PE; 47 | else 48 | val_ests(trial_choice) = val_ests(trial_choice) + alpha_neg * PE; 49 | end 50 | 51 | end 52 | 53 | % Put priors on parameters 54 | %lik= lik+log(pdf('beta', alpha_pos, 1.1,1.1)); 55 | %lik= lik+log(pdf('beta', alpha_neg, 1.1,1.1)); 56 | %lik= lik+log(pdf('gam', beta, 4.82, 0.88)); 57 | 58 | %flip sign of loglikelihood (which is negative, and we want it to be as close to 0 as possible) so we can enter it into fmincon, which searches for minimum, rather than maximum values 59 | lik = -lik; 60 | 61 | -------------------------------------------------------------------------------- /SectionD/model_recovery/model_recovery.m: -------------------------------------------------------------------------------- 1 | %% Plot model metrics and recoverability %% 2 | % Kate Nussenbaum - katenuss@nyu.edu 3 | % Flux Computational Modeling Workshop 4 | % Last edited: 8/30/21 5 | 6 | %clear everything 7 | clear; 8 | 9 | %% LOAD DATA %% 10 | load('simulated_data/200_trials_fits_10-Sep-2021'); 11 | 12 | %% CREATE CONFUSION MATRICES %% 13 | %-----------------------------------------------------------------% 14 | % STEP 1: Extract best AICs and BICs for each dataset and subject % 15 | %-----------------------------------------------------------------% 16 | %initialize matrices to store AIC and BIC values 17 | best_models_aic = []; 18 | best_models_bic = []; 19 | 20 | %loop through datasets 21 | for dataset = 1:size(model_fits, 1) 22 | dataset_aics = []; 23 | dataset_bics = []; 24 | 25 | %loop through models 26 | for model = 1:size(model_fits, 2) 27 | dataset_aics = [dataset_aics, model_fits(dataset, model).results.AIC]; 28 | [~, dataset_aic_index] = min(dataset_aics, [], 2); 29 | dataset_bics = [dataset_bics, model_fits(dataset, model).results.BIC]; 30 | [~, dataset_bic_index] = min(dataset_bics, [], 2); 31 | end 32 | best_models_aic = [best_models_aic, dataset_aic_index]; 33 | best_models_bic = [best_models_bic, dataset_bic_index]; 34 | end 35 | 36 | %-----------------------------------------------------------% 37 | % STEP 2: Create tables with simulated and recovered models % 38 | %-----------------------------------------------------------% 39 | 40 | % Get model names 41 | for dataset = 1:size(model_fits, 2) 42 | model_name{dataset} = model_fits(dataset, 1).sim_model(5:end); 43 | end 44 | 45 | %create tables 46 | aic_table = array2table(best_models_aic, 'VariableNames', model_name); 47 | aic_table_stacked = stack(aic_table, 1:size(model_fits, 2), 'NewDataVariableName','Recovered Model',... 48 | 'IndexVariableName','Simulated Model'); 49 | bic_table = array2table(best_models_bic, 'VariableNames', model_name); 50 | bic_table_stacked = stack(bic_table, 1:size(model_fits, 2), 'NewDataVariableName','Recovered Model',... 51 | 'IndexVariableName','Simulated Model'); 52 | 53 | %------------------------% 54 | % STEP 3: Create heatmap % 55 | %------------------------% 56 | 57 | % Plot 58 | figure; 59 | subplot(1,2,1) 60 | h = heatmap(aic_table_stacked, 'Simulated Model', 'Recovered Model'); 61 | h.YDisplayLabels = model_name; 62 | title('AIC'); 63 | set(gca,'FontSize',14) 64 | colorbar off 65 | subplot(1,2,2) 66 | h = heatmap(bic_table_stacked, 'Simulated Model', 'Recovered Model'); 67 | h.YDisplayLabels = model_name; 68 | title('BIC'); 69 | set(gca,'FontSize',14) 70 | colorbar off 71 | 72 | %% PLOT MEAN AND MEDIAN AIC AND BIC VALUES %% 73 | %-------------------------------------------------------------------------------% 74 | % STEP 1: Compute mean and median AIC and BIC values for each dataset and model % 75 | %-------------------------------------------------------------------------------% 76 | 77 | %initialize matrices 78 | mean_aic = NaN(size(model_fits)); 79 | mean_negloglik = NaN(size(model_fits)); 80 | num_params = NaN(size(model_fits)); 81 | med_aic = NaN(size(model_fits)); 82 | mean_bic = NaN(size(model_fits)); 83 | med_bic = NaN(size(model_fits)); 84 | 85 | for dataset = 1:size(model_fits, 1) 86 | for model = 1:size(model_fits, 2) 87 | mean_aic(dataset, model) = mean(model_fits(dataset, model).results.AIC); 88 | mean_negloglik(dataset, model) = mean(model_fits(dataset, model).results.negloglik); 89 | num_params(dataset, model) = size(model_fits(dataset, model).results.params, 2); 90 | med_aic(dataset, model) = median(model_fits(dataset, model).results.AIC); 91 | mean_bic(dataset, model) = mean(model_fits(dataset, model).results.BIC); 92 | med_bic(dataset, model) = median(model_fits(dataset, model).results.BIC); 93 | end 94 | end 95 | 96 | %----------------------------------------------% 97 | % STEP 2: % Plot mean and median AICs and BICs % 98 | %----------------------------------------------% 99 | 100 | figure; 101 | subplot(2,2,1) 102 | b = bar(mean_aic, 'EdgeColor','black', 'LineWidth', 1); 103 | set(gca, 'xticklabel', model_name); 104 | set(gca,'FontName','Helvetica','FontSize',16); 105 | xlabel('Simulated Model','FontSize',18); 106 | ylabel('Mean AIC', 'FontSize', 18); 107 | leg = legend(model_name, 'Location', 'northeastoutside'); 108 | title(leg,'Fit Model'); 109 | 110 | subplot(2,2,2) 111 | b = bar(med_aic, 'EdgeColor','black', 'LineWidth', 1); 112 | set(gca, 'xticklabel', model_name); 113 | set(gca,'FontName','Helvetica','FontSize',16); 114 | xlabel('Simulated Model','FontSize',18); 115 | ylabel('Median AIC', 'FontSize', 18); 116 | leg = legend(model_name, 'Location', 'northeastoutside'); 117 | title(leg,'Fit Model'); 118 | 119 | subplot(2,2,3) 120 | b = bar(mean_bic, 'EdgeColor','black', 'LineWidth', 1); 121 | set(gca, 'xticklabel',model_name); 122 | set(gca,'FontName','Helvetica','FontSize',16); 123 | xlabel('Simulated Model','FontSize',18); 124 | ylabel('Mean BIC', 'FontSize', 18); 125 | leg = legend(model_name, 'Location', 'northeastoutside'); 126 | title(leg,'Fit Model'); 127 | 128 | subplot(2,2,4) 129 | b = bar(med_bic, 'EdgeColor','black', 'LineWidth', 1); 130 | set(gca, 'xticklabel',model_name); 131 | set(gca,'FontName','Helvetica','FontSize',16); 132 | xlabel('Simulated Model','FontSize',18); 133 | ylabel('Median BIC', 'FontSize', 18); 134 | leg = legend(model_name, 'Location', 'northeastoutside'); 135 | title(leg,'Fit Model'); 136 | 137 | 138 | %% 139 | %--------------------------------------------------------------% 140 | % For Flux Workshop: Plot mean AICs, decomposed into two terms % 141 | %--------------------------------------------------------------% 142 | 143 | figure; 144 | subplot(1,3,1) 145 | b = bar(2*mean_negloglik(1, 1:3),'FaceColor', [153,51,102]/255, 'EdgeColor','black', 'LineWidth', 1); 146 | set(gca, 'xticklabel', model_name(1:3)); 147 | set(gca,'FontName','Helvetica','FontSize',16); 148 | ylim([min(2*mean_negloglik(1, 1:3)) - 5, max(2*mean_negloglik(1, 1:3)) + 10]) 149 | xlabel('Model','FontSize',18); 150 | ylabel('-2ln(L)', 'FontSize', 18); 151 | 152 | subplot(1,3,2) 153 | b = bar(2*num_params(1, 1:3), 'FaceColor', [211,211,211]/255, 'EdgeColor','black', 'LineWidth', 1); 154 | set(gca, 'xticklabel', model_name(1:3)); 155 | set(gca,'FontName','Helvetica','FontSize',16); 156 | ylim([0, 2*max(num_params(1, 1:3)) + 1]) 157 | xlabel('Model','FontSize',18); 158 | ylabel('2k', 'FontSize', 18); 159 | 160 | subplot(1,3,3) 161 | b = bar([2*mean_negloglik(1, 1:3); 2*num_params(1, 1:3)]', 'stacked' ,'EdgeColor','black', 'LineWidth', 1,'FaceColor','flat'); 162 | b(1).CData = [153,51,102]/255; 163 | b(2).CData = [211,211,211]/255; 164 | set(gca, 'xticklabel', model_name(1:3)); 165 | set(gca,'FontName','Helvetica','FontSize',16); 166 | ylim([min(2*mean_negloglik(1, 1:3)) - 5, max(2*mean_negloglik(1, 1:3)) + 10]) 167 | xlabel('Model','FontSize',18); 168 | ylabel('AIC', 'FontSize', 18); 169 | 170 | 171 | 172 | 173 | 174 | 175 | 176 | -------------------------------------------------------------------------------- /SectionD/model_recovery/models/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/models/.DS_Store -------------------------------------------------------------------------------- /SectionD/model_recovery/models/sim_decay.m: -------------------------------------------------------------------------------- 1 | 2 | function [blocks, choices, rewards] = sim_decay(task_struct, params) 3 | 4 | %get task structure data 5 | num_blocks = task_struct.num_blocks; 6 | num_block_trials = task_struct.num_block_trials; 7 | reward_probs = task_struct.reward_probs; 8 | 9 | %get params 10 | tau = params(1); 11 | alpha_init = params(2); 12 | eta = params(3); %decay rate 13 | 14 | %initialize choices 15 | choices = zeros(num_blocks * num_block_trials, 1); 16 | rewards = zeros(num_blocks * num_block_trials, 1); 17 | blocks = zeros(num_blocks * num_block_trials, 1); 18 | 19 | %loop through blocks and trials 20 | for block = 1:num_blocks 21 | val_ests = [.5 .5]; % Initialize values for each subject at beginning of block 22 | 23 | %loop through trials 24 | for trial = 1:num_block_trials 25 | 26 | % Determine choice probabilities 27 | ev = exp(val_ests./tau); %multiply inverse temperature * value estimates and exponentiate 28 | sev = sum(ev); 29 | choice_probs = ev/sev; %divide values by sum of all values so the probabilities sum to 1 30 | 31 | % Coin flip to determine which bandit to choose 32 | if rand(1) < choice_probs(1) 33 | choice = 1; 34 | else 35 | choice = 2; 36 | end 37 | 38 | %Coin flip to determine reward outcome 39 | if rand(1) < reward_probs(choice) 40 | reward = 1; 41 | else 42 | reward = 0; 43 | end 44 | 45 | % Compute prediction error based on reward 46 | pe = reward - val_ests(choice); 47 | 48 | %determine learning rate for each trial 49 | if trial == 1 50 | alpha = alpha_init; 51 | else 52 | alpha = alpha_init/(1+eta*(trial-1)); 53 | end 54 | 55 | % Update value estimate based on learning rate * prediction error 56 | val_ests(choice) = val_ests(choice) + alpha * pe; 57 | 58 | %save trial information 59 | choices(trial + (block - 1) * num_block_trials) = choice; 60 | rewards(trial + (block - 1) * num_block_trials) = reward; 61 | blocks(trial + (block - 1) * num_block_trials) = block; 62 | end 63 | 64 | end 65 | 66 | 67 | 68 | 69 | 70 | -------------------------------------------------------------------------------- /SectionD/model_recovery/models/sim_null.m: -------------------------------------------------------------------------------- 1 | 2 | function [blocks, choices, rewards] = sim_null(task_struct) 3 | 4 | %get task structure data 5 | num_blocks = task_struct.num_blocks; 6 | num_block_trials = task_struct.num_block_trials; 7 | reward_probs = task_struct.reward_probs; 8 | 9 | 10 | %initialize choices 11 | choices = zeros(num_blocks * num_block_trials, 1); 12 | rewards = zeros(num_blocks * num_block_trials, 1); 13 | blocks = zeros(num_blocks * num_block_trials, 1); 14 | 15 | %loop through blocks and trials 16 | for block = 1:num_blocks 17 | choice_probs = [.5 .5]; % Initialize values for each subject at beginning of block 18 | 19 | %loop through trials 20 | for trial = 1:num_block_trials 21 | 22 | % Coin flip to determine which bandit to choose 23 | if rand(1) < choice_probs(1) 24 | choice = 1; 25 | else 26 | choice = 2; 27 | end 28 | 29 | %Coin flip to determine reward outcome 30 | if rand(1) < reward_probs(choice) 31 | reward = 1; 32 | else 33 | reward = 0; 34 | end 35 | 36 | %save trial information 37 | choices(trial + (block - 1) * num_block_trials) = choice; 38 | rewards(trial + (block - 1) * num_block_trials) = reward; 39 | blocks(trial + (block - 1) * num_block_trials) = block; 40 | end 41 | end 42 | 43 | end 44 | 45 | 46 | 47 | 48 | 49 | -------------------------------------------------------------------------------- /SectionD/model_recovery/models/sim_oneLR.m: -------------------------------------------------------------------------------- 1 | 2 | function [blocks, choices, rewards] = sim_oneLR(task_struct, params) 3 | 4 | %get task structure data 5 | num_blocks = task_struct.num_blocks; 6 | num_block_trials = task_struct.num_block_trials; 7 | reward_probs = task_struct.reward_probs; 8 | 9 | %get params 10 | tau = params(1); 11 | alpha = params(2); 12 | 13 | %initialize choices 14 | choices = zeros(num_blocks * num_block_trials, 1); 15 | rewards = zeros(num_blocks * num_block_trials, 1); 16 | blocks = zeros(num_blocks * num_block_trials, 1); 17 | 18 | %loop through blocks and trials 19 | for block = 1:num_blocks 20 | val_ests = [.5 .5]; % Initialize values for each subject at beginning of block 21 | 22 | %loop through trials 23 | for trial = 1:num_block_trials 24 | 25 | % Determine choice probabilities 26 | ev = exp(val_ests./tau); %multiply inverse temperature * value estimates and exponentiate 27 | sev = sum(ev); 28 | choice_probs = ev/sev; %divide values by sum of all values so the probabilities sum to 1 29 | 30 | % Coin flip to determine which bandit to choose 31 | if rand(1) < choice_probs(1) 32 | choice = 1; 33 | else 34 | choice = 2; 35 | end 36 | 37 | %Coin flip to determine reward outcome 38 | if rand(1) < reward_probs(choice) 39 | reward = 1; 40 | else 41 | reward = 0; 42 | end 43 | 44 | % Compute prediction error based on reward 45 | pe = reward - val_ests(choice); 46 | 47 | % Update value estimate based on learning rate * prediction error 48 | val_ests(choice) = val_ests(choice) + alpha * pe; 49 | 50 | %save trial information 51 | choices(trial + (block - 1) * num_block_trials) = choice; 52 | rewards(trial + (block - 1) * num_block_trials) = reward; 53 | blocks(trial + (block - 1) * num_block_trials) = block; 54 | end 55 | 56 | end 57 | 58 | 59 | 60 | 61 | 62 | -------------------------------------------------------------------------------- /SectionD/model_recovery/models/sim_twoLR.m: -------------------------------------------------------------------------------- 1 | 2 | function [blocks, choices, rewards] = sim_twoLR(task_struct, params) 3 | 4 | %get task structure data 5 | num_blocks = task_struct.num_blocks; 6 | num_block_trials = task_struct.num_block_trials; 7 | reward_probs = task_struct.reward_probs; 8 | 9 | %get params 10 | tau = params(1); 11 | alpha_pos = params(2); 12 | alpha_neg = params(3); 13 | 14 | %initialize choices 15 | choices = zeros(num_blocks * num_block_trials, 1); 16 | rewards = zeros(num_blocks * num_block_trials, 1); 17 | blocks = zeros(num_blocks * num_block_trials, 1); 18 | 19 | %loop through blocks and trials 20 | for block = 1:num_blocks 21 | val_ests = [.5 .5]; % Initialize values for each subject at beginning of block 22 | 23 | %loop through trials 24 | for trial = 1:num_block_trials 25 | 26 | % Determine choice probabilities 27 | ev = exp(val_ests./tau); %multiply inverse temperature * value estimates and exponentiate 28 | sev = sum(ev); 29 | choice_probs = ev/sev; %divide values by sum of all values so the probabilities sum to 1 30 | 31 | % Coin flip to determine which bandit to choose 32 | if rand(1) < choice_probs(1) 33 | choice = 1; 34 | else 35 | choice = 2; 36 | end 37 | 38 | %Coin flip to determine reward outcome 39 | if rand(1) < reward_probs(choice) 40 | reward = 1; 41 | else 42 | reward = 0; 43 | end 44 | 45 | % Compute prediction error based on reward 46 | pe = reward - val_ests(choice); 47 | 48 | % Update value estimate based on learning rate * prediction error 49 | if pe > 0 50 | val_ests(choice) = val_ests(choice) + alpha_pos * pe; 51 | else 52 | val_ests(choice) = val_ests(choice) + alpha_neg * pe; 53 | end 54 | 55 | 56 | %save trial information 57 | choices(trial + (block - 1) * num_block_trials) = choice; 58 | rewards(trial + (block - 1) * num_block_trials) = reward; 59 | blocks(trial + (block - 1) * num_block_trials) = block; 60 | end 61 | 62 | end 63 | 64 | 65 | 66 | 67 | 68 | -------------------------------------------------------------------------------- /SectionD/model_recovery/parameter_recovery.m: -------------------------------------------------------------------------------- 1 | %% Plot parameter recoverability %% 2 | % Kate Nussenbaum - katenuss@nyu.edu 3 | % Flux Computational Modeling Workshop 4 | % Last edited: 8/30/21 5 | 6 | clear; 7 | 8 | %% LOAD SIMULATED DATA AND MODEL FITS %% 9 | 10 | %simulated data 11 | load('simulated_data/data_30-Aug-2021'); 12 | 13 | %model fits 14 | load('simulated_data/fits_30-Aug-2021'); 15 | 16 | %% GET SIMULATED AND RECOVERED PARAMETERS FROM EACH MODEL %% 17 | for m = 1:length(sim_data) 18 | sim_params = zeros(size(model_fits(m,m).results.params)); 19 | fit_params = zeros(size(model_fits(m,m).results.params)); 20 | for s = 1:length(sim_data(1).sub_data) 21 | if ~isempty(sim_data(m).sub_data(s).params) 22 | sim_params(s, :) = sim_data(m).sub_data(s).params; 23 | fit_params(s, :) = model_fits(m, m).results.params(s, :); 24 | end 25 | end 26 | params(m).sim = sim_params; 27 | params(m).fit = fit_params; 28 | end 29 | 30 | %% PLOT CORRELATIONS BETWEEN SIMULATED AND RECOVERED PARAMETERS %% 31 | 32 | for m = 1:length(params) 33 | if ~isempty(params) 34 | figure; % new figure for each model 35 | n_params = sim_data(m).n_params; %get number of parameters 36 | param_names = sim_data(m).param_names; %get parameter names 37 | 38 | for p = 1:n_params 39 | subplot(1, n_params, p); %new subplot for each parameter 40 | 41 | %make scatter plot 42 | scatter(params(m).sim(:, p), params(m).fit(:, p), 10, 'MarkerFaceColor', [102 102 255]./255, 'MarkerEdgeColor', 'k'); 43 | 44 | %add linear regression line 45 | hold on; 46 | P = polyfit(params(m).sim(:, p), params(m).fit(:, p), 1); 47 | yfit = P(1)*params(m).sim(:, p)+P(2); 48 | plot(params(m).sim(:, p),yfit,'k-.', 'LineWidth', 3); 49 | 50 | %add parameter name to plot 51 | title([param_names(p), round(P(1), 2)], 'Interpreter', 'none'); 52 | set(gca, 'FontSize', 12); 53 | end 54 | 55 | %add model name to plot 56 | sgtitle(sim_data(m).function(5:end), 'FontSize', 20); 57 | end 58 | 59 | end 60 | 61 | -------------------------------------------------------------------------------- /SectionD/model_recovery/simulate_choice_data.m: -------------------------------------------------------------------------------- 1 | %% Generate simulated choice data %% 2 | % Kate Nussenbaum - katenuss@nyu.edu 3 | % Flux Computational Modeling Workshop 4 | % Last edited: 8/30/21 5 | 6 | %clear everything 7 | clear; 8 | 9 | %reset random stream 10 | rng('shuffle'); 11 | 12 | %load path to models 13 | addpath('models'); 14 | 15 | %% VARIABLES TO MODIFY %% 16 | % Data file name 17 | save_filename = ['simulated_data/200_trials_data_', date]; 18 | 19 | % Number of subjects to simulate 20 | num_subs = 1000; 21 | 22 | % Task structure 23 | task_struct.reward_probs = [.8 .2]; %reward probabilities 24 | task_struct.num_blocks = 1; %number of blocks (assumes value estimates are re-set at beginning of each block) 25 | task_struct.num_block_trials = 200; %number of trials per block 26 | 27 | %Models to simulate 28 | models = {'1LR', 'decay', 'null'}; 29 | 30 | % Note: Can also modify parameter distributions in code below 31 | 32 | %% GENERATE SIMULATED DATA %% 33 | %----------------------------------% 34 | % Loop through models and subjects % 35 | %----------------------------------% 36 | for m = 1:length(models) 37 | model_to_simulate = models{m}; 38 | 39 | %print message about which subject is being fit 40 | fprintf('Simulating model %d out of %d...\n', m, length(models)) 41 | 42 | % draw parameters for each model 43 | if strcmp(model_to_simulate, '1LR') 44 | tau = rand(1, num_subs); %between 0 and 1 45 | alpha = rand(1, num_subs); %between 0 and 1 46 | params = [tau', alpha']; 47 | param_names = {'beta', 'alpha'}; 48 | function_name = 'sim_oneLR'; 49 | elseif strcmp(model_to_simulate, '2LR') %for simulation purposes, make the learning rates different from each other 50 | tau = rand(1, num_subs); %between 0 and 1 51 | alpha_pos = .2 * rand(1, num_subs) + .3; %between .3 and 5 52 | alpha_neg = .05 * rand(1, num_subs) + .05; %between 0.05 and .1 53 | params = [tau', alpha_pos', alpha_neg']; 54 | param_names = {'beta', 'alpha_pos', 'alpha_neg'}; 55 | function_name = 'sim_twoLR'; 56 | elseif strcmp(model_to_simulate, 'null') 57 | params = []; 58 | param_names = {}; 59 | function_name = 'sim_null'; 60 | elseif strcmp(model_to_simulate, 'decay') 61 | tau = rand(1, num_subs); %between 0 and 1 62 | alpha_init = rand(1, num_subs); %between 0 and 1 63 | eta = .5 * rand(1, num_subs) + .5; %between .5 and 1 (for simulation purposes, make decay above .5) 64 | params = [tau', alpha_init', eta']; 65 | param_names = {'beta', 'alpha_init', 'eta'}; 66 | function_name = 'sim_decay'; 67 | end 68 | 69 | % convert function name into function 70 | fh = str2func(function_name); 71 | 72 | %loop through subjects and generate data 73 | for s = 1:num_subs 74 | if isempty(params) 75 | [blocks, choices, rewards] = fh(task_struct); 76 | model_data(s).params = []; 77 | else 78 | [blocks, choices, rewards] = fh(task_struct, params(s, :)); 79 | model_data(s).params = params(s, :); 80 | 81 | end 82 | 83 | %for each subject, save the generating parameters, choices, and 84 | %rewards 85 | model_data(s).choices = choices; 86 | model_data(s).rewards = rewards; 87 | model_data(s).blocks = blocks; 88 | end 89 | 90 | %for each model, save the simulated subject data, name of the model, 91 | %number of parameters, and parameter names 92 | sim_data(m).sub_data = model_data; 93 | sim_data(m).function = function_name; 94 | sim_data(m).n_params = size(params, 2); 95 | sim_data(m).param_names = param_names; 96 | 97 | end 98 | 99 | %---------------------% 100 | % Save simulated data % 101 | %---------------------% 102 | save(save_filename, 'sim_data'); 103 | -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/.DS_Store -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/200_trials_data_10-Sep-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/200_trials_data_10-Sep-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/200_trials_fits_10-Sep-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/200_trials_fits_10-Sep-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/20_trials_data_10-Sep-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/20_trials_data_10-Sep-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/20_trials_fits_10-Sep-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/20_trials_fits_10-Sep-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/data_10-Sep-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/data_10-Sep-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/data_27-Aug-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/data_27-Aug-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/data_28-Aug-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/data_28-Aug-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/data_30-Aug-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/data_30-Aug-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/fits_10-Sep-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/fits_10-Sep-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/fits_27-Aug-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/fits_27-Aug-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/fits_28-Aug-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/fits_28-Aug-2021.mat -------------------------------------------------------------------------------- /SectionD/model_recovery/simulated_data/fits_30-Aug-2021.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/model_recovery/simulated_data/fits_30-Aug-2021.mat -------------------------------------------------------------------------------- /SectionD/parameter_recovery/Palminteri_Wyart_Koechlin_Trends_2017.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/parameter_recovery/Palminteri_Wyart_Koechlin_Trends_2017.pdf -------------------------------------------------------------------------------- /SectionD/parameter_recovery/alpha0.5_100trials.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/parameter_recovery/alpha0.5_100trials.png -------------------------------------------------------------------------------- /SectionD/parameter_recovery/alpha0.5_300trials.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/parameter_recovery/alpha0.5_300trials.png -------------------------------------------------------------------------------- /SectionD/parameter_recovery/decay_learningrate.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/parameter_recovery/decay_learningrate.png -------------------------------------------------------------------------------- /SectionD/parameter_recovery/model_falsification.m: -------------------------------------------------------------------------------- 1 | % Model falsification 2 | % Script to simulate the behaviour of a model with fixed and decaying learning rate and compare to the "real" data on the probability to switch after a negative prediction error at the beginning and at the end of the session 3 | % prediction is that subjects stabilize their choices at the end of the 4 | % session -> ignore the irrelevant occasional bad outcomes from a good deck 5 | 6 | 7 | % The model with the fixed learning rate won't be able to match this 8 | % pattern. 9 | % NB: we assume the softmax is the same in both models. 10 | 11 | 12 | % Vasilisa Skvortsova, 08/2021 for FLUX 13 | 14 | 15 | clear all; 16 | close all; 17 | clc; 18 | 19 | 20 | %% PART 1: simulate subject's data using a model with the decaying learning rate: in the real life you of course use the true subjects data in the task 21 | 22 | % Generative model 23 | cfg_data.nsub = 30; 24 | cfg_data.p = 0.8; % 80 20 25 | cfg_data.nblck = 1; % assuming 1 block 26 | % fix learning rate for chosen and unchosen option 27 | cfg_data.alphach = 0.7; % learning rate for the chosen option 28 | cfg_data.eta = 0.02; % an exponentail decay rate for the learning rate 29 | cfg_data.tau = 0.2; % an exponentail decay rate for the learning rate 30 | cfg_data.alphauch = 0; % .4; % learning rate for the unchosen option 31 | cfg_data.ntrl = 300; % assume we have 2 blocks of 60 trials each with 2 reversals within each block (for simplicity in the middle of the block). 32 | 33 | true_data = simulate_data(cfg_data); 34 | 35 | %% PART 2 Simulate performance of : 1) model with the fixed learning rate 2) model with the decaying learning rate 36 | clc; 37 | 38 | % Model 1 with fixed learning rate 39 | cfg_m1.nsub = 30; 40 | cfg_m1.p = 0.8; % 80 20 41 | cfg_m1.nblck = 1; % assuming 1 block 42 | % fix learning rate for chosen and unchosen option 43 | cfg_m1.alphach = 0.6; % learning rate for the chosen option 44 | cfg_m1.eta = 0; % an exponentail decay rate for the learning rate 45 | cfg_m1.tau = 0.2; % an exponentail decay rate for the learning rate 46 | cfg_m1.alphauch = 0; % .4; % learning rate for the unchosen option 47 | cfg_m1.ntrl = 300; % assume we have 2 blocks of 60 trials each with 2 reversals within each block (for simplicity in the middle of the block). 48 | 49 | m1_data = simulate_data(cfg_m1); 50 | 51 | 52 | % Model 2 with decaying learning rate 53 | cfg_m2.nsub = 30; 54 | cfg_m2.p = 0.8; % 80 20 55 | cfg_m2.nblck = 1; % assuming 1 block 56 | % fix learning rate for chosen and unchosen option 57 | cfg_m2.alphach = 0.8; % learning rate for the chosen option 58 | cfg_m2.eta = 0.02; % an exponentail decay rate for the learning rate 59 | cfg_m2.tau = 0.2; % an exponentail decay rate for the learning rate 60 | cfg_m2.alphauch = 0; % .4; % learning rate for the unchosen option 61 | cfg_m2.ntrl = 300; % assume we have 2 blocks of 60 trials each with 2 reversals within each block (for simplicity in the middle of the block). 62 | 63 | m2_data = simulate_data(cfg_m2); 64 | 65 | 66 | 67 | %% PART 3: Compare model performance to true data on the benchmark: Probability to switch after a negative prediction error at the beginning and at the end of the learning session 68 | clc; 69 | % compute probability to switch as a function of small/large prediction 70 | % error 71 | 72 | switch_data = compute_switch(true_data); 73 | switch_m1 = compute_switch(m1_data); 74 | switch_m2 = compute_switch(m2_data); 75 | 76 | %% PLOT data and model performances 77 | clc; 78 | close all; 79 | 80 | pbar = 1.0; 81 | figure('Color','white','Name','Model falsification'); 82 | set(gcf, 'Position', [100, 100, 600, 600]) 83 | 84 | 85 | % plot model frequencies for high-scoring subjects 86 | xavg_data = cat(2,mean(switch_data.pe_switch1,1),mean(switch_data.pe_switch2,1)); 87 | xstd_data = cat(2,std(switch_data.pe_switch1,1),std(switch_data.pe_switch2,1))./sqrt(cfg_data.nsub); 88 | 89 | xavg_data = repmat(xavg_data,2,1); 90 | xstd_data = repmat(xstd_data,2,1); 91 | 92 | 93 | xavg_1 = cat(2,mean(switch_m1.pe_switch1,1),mean(switch_m1.pe_switch2,1)); 94 | xstd_1 = cat(2,std(switch_m1.pe_switch1,1),std(switch_m1.pe_switch2,1))./sqrt(cfg_m1.nsub); 95 | xavg_2 = cat(2,mean(switch_m2.pe_switch1,1),mean(switch_m2.pe_switch2,1)); 96 | xstd_2 = cat(2,std(switch_m2.pe_switch1,1),std(switch_m2.pe_switch2,1))./sqrt(cfg_m2.nsub); 97 | 98 | xavg = cat(1,xavg_1,xavg_2); 99 | xstd = cat(1,xstd_1,xstd_2); 100 | 101 | 102 | ngrp = size(xavg,1); 103 | nbar = size(xavg,2); 104 | wbar = 0.9; 105 | wgrp = min(wbar,nbar/(nbar+1.5)); 106 | 107 | subplot(1,1,1); 108 | hold on 109 | h = bar(xavg(:,:,1),0.9,'LineWidth',1); 110 | 111 | 112 | set(h(1),'FaceColor',[0.5,0.5,0.5]); 113 | set(h(2),'FaceColor',[0.875,0.875,0.875]); 114 | 115 | 116 | for ibar = 1:nbar 117 | for igrp = 1:ngrp 118 | x = igrp-wgrp/2+(2*ibar-1)*wgrp/nbar/2; 119 | plot(x*[1,1],xavg(igrp,ibar,1)+xstd(igrp,ibar,1)*[-1,+1],'k-','MarkerSize',12,'MarkerFaceColor',[1,1,1]); 120 | 121 | plot(x*[1,1],xavg_data(igrp,ibar,1),'ko','MarkerSize',12,'MarkerFaceColor',[1,1,1]); 122 | 123 | end 124 | 125 | 126 | end 127 | 128 | 129 | legend('first half','second half','','data'); 130 | 131 | xlim([0.5,2.5]); 132 | ylim([0,0.4]); 133 | set(gca,'Layer','top','Box','off','PlotBoxAspectRatio',[pbar,1,1]); 134 | set(gca,'TickDir','out','TickLength',[1,1]*0.02/max(pbar,1)); 135 | set(gca,'FontName','Helvetica','FontSize',16); 136 | set(gca,'XTick',[1,2],'XTickLabel',{'model 1','model 2'}); 137 | set(gca,'YTick',0:0.1:0.4); 138 | ylabel('$P(switch_{t}| PE_{t-1} < 0)$','Interpreter','Latex','FontSize',22); 139 | drawnow; 140 | 141 | %% === Additional functions ====== %% 142 | 143 | function out_switch = compute_switch(data) 144 | 145 | nsub = size(data,2); 146 | ntrl = numel(data(1).pe); 147 | 148 | out_switch = zeros(nsub,4); 149 | for i_s = 1:nsub 150 | 151 | % split the data into 2 parts 152 | 153 | p = data(i_s).pe; 154 | r = data(i_s).resp; 155 | 156 | pe_idx = find(p<0); 157 | 158 | % Compute the proportion of switches in part 1 and part 2: 159 | 160 | meanswitch_1 = mean(abs(diff(r(1:ntrl/2)))); 161 | meanswitch_2 = mean(abs(diff(r(ntrl/2+1:end)))); 162 | 163 | % switch after a negative pe 164 | count_switch1 = 0; 165 | count_switch2 = 0; 166 | 167 | for ii = 1:length(pe_idx) 168 | 169 | if pe_idx(ii) < ntrl 170 | 171 | if r(pe_idx(ii)+1)~=r(pe_idx(ii)) 172 | if pe_idx(ii) <=ntrl/2 % if it was a switch on the next trial 173 | count_switch1 = count_switch1 + 1; 174 | else 175 | count_switch2 = count_switch2 + 1; 176 | 177 | end 178 | 179 | end 180 | else 181 | continue 182 | end 183 | end 184 | 185 | switch_1 = mean(count_switch1)/sum(pe_idx<=ntrl/2); 186 | switch_2 = mean(count_switch2)/sum(pe_idx>ntrl/2); 187 | 188 | 189 | out_switch(i_s,1) = switch_1; 190 | out_switch(i_s,2) = switch_2; 191 | out_switch(i_s,3) = meanswitch_1; 192 | out_switch(i_s,4) = meanswitch_2; 193 | 194 | end 195 | 196 | out_switch = array2table(out_switch,'VariableNames',{'pe_switch1','pe_switch2','meanswitch1','meanswitch2'}); 197 | 198 | end 199 | 200 | 201 | function out = simulate_data(cfg) 202 | 203 | if ~isfield(cfg,'p') 204 | fprintf('No reward probability specified, assume probability of reward is 0.75\n') 205 | p = 0.75; 206 | else 207 | p = cfg.p; 208 | end 209 | 210 | 211 | if ~isfield(cfg,'tau') 212 | fprintf('No value for softmax, assuming tau = 0.1\n') 213 | tau = 0.1; 214 | else 215 | tau = cfg.tau; 216 | end 217 | 218 | 219 | if ~isfield(cfg,'alphach') 220 | fprintf('No value for learning rate chosen, assuming learnng rate chosen = 0.5\n') 221 | alphach = 0.5; 222 | else 223 | alphach = cfg.alphach; 224 | end 225 | 226 | 227 | if ~isfield(cfg,'alphauch') 228 | fprintf('No value for learning rate unchosen, assuming no update for the unchosen option\n') 229 | alphauch = 0; 230 | else 231 | alphauch = cfg.alphauch; 232 | end 233 | 234 | 235 | if ~isfield(cfg,'eta') 236 | fprintf('No value for decaying learning rate assume learning rate is fixed\n') 237 | eta = 0; 238 | else 239 | eta = cfg.eta; 240 | end 241 | 242 | 243 | if ~isfield(cfg,'nsub') 244 | fprintf('Assuming 1 subject\n') 245 | nsub = 1; 246 | else 247 | nsub = cfg.nsub; 248 | end 249 | 250 | 251 | if ~isfield(cfg,'ntrl') 252 | fprintf('Assuming 120 trials\n') 253 | 254 | ntrl = 120; 255 | else 256 | ntrl = cfg.ntrl; 257 | end 258 | 259 | 260 | if ~isfield(cfg,'nblck') 261 | fprintf('Assuming 2 blocks\n') 262 | 263 | nblck = 2; 264 | else 265 | nblck = cfg.nblck; 266 | end 267 | 268 | 269 | ntrl_blck = ntrl/nblck; 270 | 271 | 272 | 273 | for i_s = 1:nsub 274 | 275 | fprintf('Simulating subject %d\n',i_s) 276 | % create binary rewards with reward probability p 277 | rew1 = []; 278 | rew2 = []; 279 | 280 | for i_b = 1:nblck 281 | r1 = binornd(1,p,1,ntrl_blck); 282 | r2 = binornd(1,1-p,1,ntrl_blck); 283 | 284 | if mod(i_b,2) == 1 285 | rew1 = cat(2,rew1,r1); 286 | rew2 = cat(2,rew2,r2); 287 | 288 | else 289 | rew1 = cat(2,rew1,r2); 290 | rew2 = cat(2,rew2,r1); 291 | end 292 | end 293 | 294 | rewards = cat(1,rew1,rew2); 295 | 296 | [resp,per] = get_simulate(rewards,alphach,alphauch,tau,eta); 297 | 298 | out(i_s).resp = resp; 299 | out(i_s).rewards = rewards; 300 | out(i_s).pe = per; 301 | 302 | end 303 | end 304 | 305 | % simulate agent with specific learning paramaters 306 | 307 | function [resp,per] = get_simulate(rewards,alphach,alphauch,tau,eta) 308 | 309 | % compute Q-values 310 | ntrl = size(rewards,2); 311 | q = nan(ntrl,2); 312 | resp = nan(ntrl,1); 313 | 314 | q(1,:) = 0.5; 315 | vm = [0.5,0.5]; % cached value for each bandit 316 | 317 | % first choice is random 318 | resp(1) = double(rand>0.5)+1; % between 1 and 2 319 | 320 | 321 | alpha_decay = nan(1,ntrl); 322 | alpha_decay(1) = alphach; 323 | per = nan(1,ntrl); 324 | per(1) = 0; 325 | 326 | 327 | for itrl = 2:ntrl 328 | 329 | r = resp(itrl-1); 330 | 331 | 332 | vm(r) = rewards(r,itrl-1); % update cached value for chosen bandit 333 | vm(3-r) = 1 - vm(r); % assume anticorrelation 334 | 335 | % compute the the decaying learning rate 336 | alpha_decay(itrl) = alphach/(1+eta*(itrl-1)); 337 | 338 | % update the Q-values 339 | per(itrl) = vm(r)-q(itrl-1,r); 340 | 341 | q(itrl,r) = q(itrl-1,r)+alpha_decay(itrl)*(vm(r)-q(itrl-1,r)); 342 | 343 | q(itrl,3-r) = q(itrl-1,3-r)+alphauch*(vm(3-r)-q(itrl-1,3-r)); % alpha unchosen is always 0 in this example 344 | 345 | 346 | deltaQ = q(itrl,1) - q(itrl,2); % difference in Q values between options 1 and option 2 347 | proba = 1./(1+exp(-deltaQ/tau)); % softmax for 2 options 348 | 349 | resp(itrl) = (1-sampleFromArbitraryP([proba,1-proba],[1,0]',1))+1; 350 | 351 | end 352 | 353 | 354 | % figure('Color','white'); 355 | % plot(1:ntrl, alpha_decay,'LineWidth',2.0,'Color','r'); 356 | % ylabel('decaying learning rate ${\alpha}$','Interpreter','Latex','FontSize',20); 357 | % xlabel('Trials','FontSize',18); 358 | % set(gca,'Layer','top','Box','off'); 359 | % set(gca,'TickDir','out','TickLength',[1,1]*0.02); 360 | % set(gca,'FontName','Helvetica','FontSize',16); 361 | 362 | 363 | 364 | end 365 | 366 | 367 | -------------------------------------------------------------------------------- /SectionD/parameter_recovery/model_falsification.mlx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/parameter_recovery/model_falsification.mlx -------------------------------------------------------------------------------- /SectionD/parameter_recovery/model_falsification.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/parameter_recovery/model_falsification.png -------------------------------------------------------------------------------- /SectionD/parameter_recovery/parameter_recovery_simulate.m: -------------------------------------------------------------------------------- 1 | % Script to simulate RL Rescorla Wagner model with different sets of paramaters 2 | 3 | % Fit the simulated data 4 | % Plot the true vs. recovered parameters 5 | 6 | % Will require general functions 7 | 8 | % Vasilisa Skvortsova, 08/2021 for FLUX 9 | 10 | clear all; 11 | close all; 12 | clc; 13 | 14 | 15 | % Simulate data for different softmax values 16 | 17 | 18 | % STEP 1: simulate the data for different softmax values and fixed learning 19 | % rate chosen and unchosen 20 | 21 | % preset the task: 2-armed bandit task with binary rewards and probability 22 | % of reward 0.75 23 | % simulate N = 30 subjects 24 | 25 | 26 | cfg_sim.nsub = 30; 27 | cfg_sim.p = 0.8; % 80 20 28 | cfg_sim.nblck = 1; % assuming 1 block 29 | % fix learning rate for chosen and unchosen option 30 | cfg_sim.alphach = 0.5; % learning rate for the chosen option 31 | cfg_sim.alphauch = 0; % .4; % learning rate for the unchosen option 32 | cfg_sim.ntrl = 100; % assume we have 2 blocks of 60 trials each with 2 reversals within each block (for simplicity in the middle of the block). 33 | 34 | 35 | sim_param = linspace(0.1,1,20); % linearly spaced values for the softmax 36 | 37 | cfg_sim.paramname = 'tau'; 38 | for idx = 1:numel(sim_param) 39 | fprintf('Simulating tau = %0.2f\n',sim_param(idx)) 40 | cfg_sim.tau = sim_param(idx); 41 | sim_out = simulate_data(cfg_sim); 42 | sim_data.(sprintf('tau%d',idx)) = sim_out; 43 | end 44 | 45 | 46 | %% PART 2 FIT simulated data 47 | clc; 48 | 49 | cfg_fit.nsub = cfg_sim.nsub; 50 | cfg_fit.alphauch = 0; % learning rate unchosen is fixed = not fitted 51 | fit_data = []; 52 | cfg_fit.nparams = 3; 53 | for idx = 1:numel(sim_param) 54 | fprintf('Recovery for tau = %0.2f\n',sim_param(idx)) 55 | 56 | 57 | params = nan(cfg_fit.nsub,cfg_fit.nparams); 58 | 59 | for i_s = 1:cfg_fit.nsub 60 | 61 | fprintf('Fitting subject %d\n',i_s) 62 | cfg_fit.resp = sim_data.(sprintf('tau%d',idx))(i_s).resp; 63 | cfg_fit.rewards = sim_data.(sprintf('tau%d',idx))(i_s).rewards; 64 | 65 | [out_fit] = fit_mle(cfg_fit); 66 | params(i_s,:) = cat(2,out_fit.alphach,out_fit.alphauch,out_fit.tau); 67 | end 68 | 69 | fit_data = cat(1,fit_data,cat(2,mean(params,1),std(params,1)./sqrt(cfg_fit.nsub))); 70 | end 71 | 72 | % convert fitted results to the table 73 | fitTable = array2table(fit_data,'VariableNames',{'alphach','alphauch','tau',... 74 | 'alphach_sem','alphauch_sem','tau_sem'}); 75 | 76 | %% Plot the parameter recovery 77 | clc; 78 | close all; 79 | 80 | pbar = 1.0; 81 | figure('Color','white','Name',sprintf('Parameter recovery for %s',cfg_sim.paramname)); 82 | set(gcf, 'Position', [100, 100, 1000, 600]) 83 | hold on 84 | subplot(1,2,1); 85 | 86 | true = sim_param; 87 | recovered = fitTable.tau; 88 | recovered_sem = fitTable.tau_sem; 89 | 90 | h1 = plot(true,recovered,'o','MarkerSize',12); hold on; 91 | h2 = errorbar(true,recovered,recovered_sem,'LineStyle','None','LineWidth',1.0); 92 | h3 = plot(true,true,'--','LineWidth',0.75); 93 | 94 | 95 | min_idx = find(abs((sim_param-0.3)) == min(abs(sim_param-0.3))); 96 | h4 = plot(true(min_idx),recovered(min_idx),'o','MarkerSize',18,'LineWidth',2); 97 | 98 | 99 | set(h1(1),'MarkerFaceColor',[153,51,102]/255); 100 | set(h2(1),'Color',[0.5,0.5,0.5]); 101 | set(h3(1),'Color',[0.5,0.5,0.5]); 102 | set(h4(1),'MarkerEdgeColor',[0.75,0.75,0.75]); 103 | 104 | 105 | h2.CapSize = 0; 106 | 107 | text(true(min_idx),sim_param(min_idx)-0.06,'\uparrow','FontSize',16) 108 | text(true(min_idx),sim_param(min_idx)-0.12,'true group \tau','FontSize',16) 109 | 110 | 111 | 112 | xlim([0,max(true)+0.25]); 113 | ylim([0,max(true)+0.25]); 114 | % set(gca,'Layer','top','Box','off','PlotBoxAspectRatio',[pbar,1,1]); 115 | 116 | set(gca,'Layer','top','Box','off'); 117 | 118 | set(gca,'TickDir','out','TickLength',[1,1]*0.02/max(pbar,1)); 119 | set(gca,'FontName','Helvetica','FontSize',16); 120 | xlabel('true softmax temperature ${\tau}$','Interpreter','Latex','FontSize',26); 121 | ylabel('recovered softmax temperature ${\tau}$','Interpreter','Latex','FontSize',26); 122 | axis square 123 | 124 | 125 | subplot(1,2,2); 126 | 127 | true = sim_param; 128 | recovered = fitTable.alphach; 129 | 130 | recovered_sem = fitTable.alphach_sem; 131 | 132 | h1 = plot(true,recovered,'o','MarkerSize',12); hold on; 133 | h2 = errorbar(true,recovered,recovered_sem,'LineStyle','None','LineWidth',1.0); 134 | h3 = yline(cfg_sim.alphach,'--','LineWidth',1.0); 135 | h4 = plot(true(min_idx),recovered(min_idx),'o','MarkerSize',18,'LineWidth',2); 136 | h5 = plot(0,cfg_sim.alphach,'o','MarkerSize',14,'LineWidth',2); 137 | 138 | set(h3(1),'Color',[0.5,0.5,0.5]); 139 | set(h1(1),'MarkerFaceColor',[153,51,102]/255); 140 | set(h2(1),'Color',[0.5,0.5,0.5]); 141 | set(h4(1),'MarkerEdgeColor',[0.75,0.75,0.75]); 142 | set(h5(1),'MarkerFaceColor',[0.75,0.75,0.75]); 143 | 144 | h2.CapSize = 0; 145 | 146 | text(true(min_idx),cfg_sim.alphach-0.05,'\uparrow','FontSize',18) 147 | text(true(min_idx),cfg_sim.alphach-0.1,'true group \tau','FontSize',18) 148 | 149 | % text(0,cfg_sim.alphach+0.06,'\downarrow true \alpha','FontSize',18) 150 | text(0,cfg_sim.alphach-0.04,'\uparrow','FontSize',18) 151 | text(0,cfg_sim.alphach-0.1,'true \alpha','FontSize',18) 152 | 153 | xlim([0,max(true)]); 154 | ylim([0,max(true)]); 155 | % set(gca,'Layer','top','Box','off','PlotBoxAspectRatio',[pbar,1,1]); 156 | set(gca,'Layer','top','Box','off'); 157 | 158 | set(gca,'TickDir','out','TickLength',[1,1]*0.02/max(pbar,1)); 159 | set(gca,'FontName','Helvetica','FontSize',16); 160 | xlabel('true softmax temperature ${\tau}$','Interpreter','Latex','FontSize',26); 161 | ylabel('recovered learning rate chosen ${\alpha}$','Interpreter','Latex','FontSize',26); 162 | axis square 163 | 164 | drawnow; 165 | 166 | %% === Additional functions ====== %% 167 | function out = simulate_data(cfg) 168 | 169 | if ~isfield(cfg,'p') 170 | fprintf('No reward probability specified, assume probability of reward is 0.75\n') 171 | p = 0.75; 172 | else 173 | p = cfg.p; 174 | end 175 | 176 | 177 | if ~isfield(cfg,'tau') 178 | fprintf('No value for softmax, assuming tau = 0.1\n') 179 | tau = 0.1; 180 | else 181 | tau = cfg.tau; 182 | end 183 | 184 | 185 | if ~isfield(cfg,'alphach') 186 | fprintf('No value for learning rate chosen, assuming learnng rate chosen = 0.5\n') 187 | alphach = 0.5; 188 | else 189 | alphach = cfg.alphach; 190 | end 191 | 192 | 193 | if ~isfield(cfg,'alphauch') 194 | fprintf('No value for learning rate unchosen, assuming no update for the unchosen option\n') 195 | alphauch = 0; 196 | else 197 | alphauch = cfg.alphauch; 198 | end 199 | 200 | if ~isfield(cfg,'nsub') 201 | fprintf('Assuming 1 subject\n') 202 | nsub = 1; 203 | else 204 | nsub = cfg.nsub; 205 | end 206 | 207 | 208 | if ~isfield(cfg,'ntrl') 209 | fprintf('Assuming 120 trials\n') 210 | 211 | ntrl = 120; 212 | else 213 | ntrl = cfg.ntrl; 214 | end 215 | 216 | 217 | if ~isfield(cfg,'nblck') 218 | fprintf('Assuming 2 blocks\n') 219 | 220 | nblck = 2; 221 | else 222 | nblck = cfg.nblck; 223 | end 224 | 225 | 226 | ntrl_blck = ntrl/nblck; 227 | 228 | 229 | 230 | for i_s = 1:nsub 231 | 232 | fprintf('Simulating subject %d\n',i_s) 233 | % create binary rewards with reward probability p 234 | rew1 = []; 235 | rew2 = []; 236 | 237 | for i_b = 1:nblck 238 | 239 | r1 = zeros(1,ntrl); 240 | r2 = zeros(1,ntrl); 241 | 242 | r1(1:floor(ntrl*p)) = 1; 243 | r1(1:floor(ntrl*(1-p))) = 1; 244 | 245 | r1 = r1(randperm(length(r1))); 246 | r2 = r2(randperm(length(r2))); 247 | 248 | if mod(i_b,2) == 1 249 | rew1 = cat(2,rew1,r1); 250 | rew2 = cat(2,rew2,r2); 251 | 252 | else 253 | rew1 = cat(2,rew1,r2); 254 | rew2 = cat(2,rew2,r1); 255 | end 256 | end 257 | 258 | rewards = cat(1,rew1,rew2); 259 | 260 | resp = get_simulate(rewards,alphach,alphauch,tau); 261 | 262 | out(i_s).resp = resp; 263 | out(i_s).rewards = rewards; 264 | 265 | 266 | % figure; 267 | % plot(out.rewards(1,:),'or'); hold on; plot(out.rewards(2,:),'ob') 268 | % plot(out.resp-1,'ok'); 269 | 270 | 271 | end 272 | end 273 | 274 | % simulate agent with specific learning paramaters 275 | 276 | function [resp] = get_simulate(rewards,alphach,alphauch,tau) 277 | 278 | % compute Q-values 279 | ntrl = size(rewards,2); 280 | q = nan(ntrl,2); 281 | resp = nan(ntrl,1); 282 | 283 | q(1,:) = 0.5; 284 | vm = [0.5,0.5]; % cached value for each bandit 285 | 286 | % first choice is random 287 | resp(1) = double(rand>0.5)+1; % between 1 and 2 288 | 289 | for itrl = 2:ntrl 290 | 291 | r = resp(itrl-1); 292 | 293 | 294 | vm(r) = rewards(r,itrl-1); % update cached value for chosen bandit 295 | vm(3-r) = 1 - vm(r); % assume anticorrelation 296 | 297 | % update the Q-values 298 | q(itrl,r) = q(itrl-1,r)+alphach*(vm(r)-q(itrl-1,r)); 299 | q(itrl,3-r) = q(itrl-1,3-r)+alphauch*(vm(3-r)-q(itrl-1,3-r)); 300 | 301 | 302 | deltaQ = q(itrl,1) - q(itrl,2); % difference in Q values between options 1 and option 2 303 | proba = 1./(1+exp(-deltaQ/tau)); % softmax for 2 options 304 | 305 | resp(itrl) = (1-sampleFromArbitraryP([proba,1-proba],[1,0]',1))+1; 306 | 307 | end 308 | 309 | end 310 | 311 | 312 | function [out] = fit_mle(cfg) 313 | 314 | resp = cfg.resp; 315 | % set response lookup indices 316 | ntrl = length(resp); 317 | rewards = cfg.rewards; 318 | 319 | ipost = sub2ind([ntrl,2],(1:ntrl)',resp); 320 | 321 | % define model parameters 322 | npar = 0; 323 | pnam = {}; % parameter name 324 | psiz = []; % parameter size 325 | pini = []; % parameter initialization value 326 | pmin = []; % parameter minimum value 327 | pmax = []; % parameter maximum value 328 | 329 | 330 | % learning rate 331 | npar = npar+1; 332 | pnam{npar,1} = 'alphach'; 333 | psiz(npar,1) = 1; 334 | pini(npar,1) = 0.5; 335 | pmin(npar,1) = 0; 336 | pmax(npar,1) = 1; 337 | 338 | % learning rate unchosen 339 | npar = npar+1; 340 | pnam{npar,1} = 'alphauch'; 341 | psiz(npar,1) = 1; 342 | pini(npar,1) = 0.5; 343 | pmin(npar,1) = 0; 344 | pmax(npar,1) = 1; 345 | 346 | 347 | % softmax temperature 348 | npar = npar+1; 349 | pnam{npar,1} = 'tau'; 350 | psiz(npar,1) = 1; 351 | pini(npar,1) = 0.5; 352 | pmin(npar,1) = 0.001; 353 | pmax(npar,1) = 100; 354 | 355 | 356 | % define fixed parameters = parameters that are not fitted 357 | pfix = cell(npar,1); 358 | for i = 1:npar 359 | if isfield(cfg,pnam{i}) 360 | pfix{i} = reshape(cfg.(pnam{i}),[psiz(i),1]); 361 | end 362 | end 363 | 364 | % define free parameters to fit 365 | ifit = cell(npar,1); 366 | pfit_ini = []; 367 | pfit_min = []; 368 | pfit_max = []; 369 | n = 1; 370 | 371 | 372 | for i = 1:npar 373 | if isempty(pfix{i}) % free parameter 374 | ifit{i} = n+[1:psiz(i)]-1; 375 | pfit_ini = cat(1,pfit_ini,pini(i)*ones(psiz(i),1)); 376 | pfit_min = cat(1,pfit_min,pmin(i)*ones(psiz(i),1)); 377 | pfit_max = cat(1,pfit_max,pmax(i)*ones(psiz(i),1)); 378 | n = n+psiz(i); 379 | elseif numel(pfix{i}) ~= psiz(i) 380 | error('wrong size for fixed parameter %s!',pnam{i}); 381 | elseif any(isnan(pfix{i})) % partly fixed parameter 382 | nfit = nnz(isnan(pfix{i})); 383 | ifit{i} = n+[1:nfit]-1; 384 | pfit_ini = cat(1,pfit_ini,pini(i)*ones(nfit,1)); 385 | pfit_min = cat(1,pfit_min,pmin(i)*ones(nfit,1)); 386 | pfit_max = cat(1,pfit_max,pmax(i)*ones(nfit,1)); 387 | n = n+nfit; 388 | end 389 | end 390 | nfit = length(pfit_ini); 391 | 392 | % fit free parameters 393 | if nfit > 0 394 | pval = fmincon(@fmin,pfit_ini,[],[],[],[],pfit_min,pfit_max,[], ... 395 | optimset('Display','notify','FunValCheck','on','Algorithm','interior-point','TolX',1e-20,'MaxFunEvals',1e6)); 396 | [~,phat] = fmin(pval); 397 | else 398 | phat = pfix; 399 | end 400 | 401 | % get best-fitting parameters 402 | out = cell2struct(phat,pnam); 403 | 404 | % get quality of fit 405 | out.nfit = nfit; % number of fitted parameters 406 | out.ntrl = ntrl; % number of observations 407 | out.llh = get_llh(phat{:}); % model log-likelihood 408 | out.aic = -2*out.llh+2*nfit+2*nfit*(nfit+1)/(ntrl-nfit+1); % model AIC 409 | out.bic = -2*out.llh+nfit*log(ntrl); % model BIC 410 | 411 | [out.ppost,out.q] = get_ppost(phat{:}); 412 | 413 | function [f,pfit] = fmin(p) 414 | % create list of model parameters 415 | pfit = cell(npar,1); 416 | for i = 1:npar 417 | if isempty(pfix{i}) % free parameter 418 | pfit{i} = p(ifit{i}); 419 | elseif any(isnan(pfix{i})) % partly fixed parameter 420 | pfit{i} = pfix{i}; 421 | pfit{i}(isnan(pfit{i})) = p(ifit{i}); 422 | else % fully fixed parameter 423 | pfit{i} = pfix{i}; 424 | end 425 | end 426 | % return negative model log-likelihood 427 | f = -get_llh(pfit{:}); 428 | end 429 | 430 | function [llh,ppost] = get_llh(varargin) 431 | % get model posterior probabilities 432 | ppost = get_ppost(varargin{:}); 433 | ppost = [ppost,1-ppost]; 434 | % get model log-likelihood 435 | 436 | 437 | llh = sum(log(max(ppost(ipost),eps))); 438 | end 439 | 440 | function [ppost,q] = get_ppost(alphach,alphauch,tau) 441 | 442 | q = nan(ntrl,2); 443 | 444 | q(1,:) = 0.5; 445 | vm = [0.5,0.5]; % cached value for each bandit 446 | 447 | 448 | for itrl = 2:ntrl 449 | 450 | r = resp(itrl-1); 451 | vm(r) = rewards(r,itrl-1); % update cached value for chosen bandit 452 | vm(3-r) = 1 - vm(r); % assume anticorrelation 453 | 454 | % update the Q-values 455 | q(itrl,r) = q(itrl-1,r)+alphach*(vm(r)-q(itrl-1,r)); 456 | q(itrl,3-r) = q(itrl-1,3-r)+alphauch*(vm(3-r)-q(itrl-1,3-r)); 457 | 458 | end 459 | 460 | ppost = 1./(1+exp(-(q(:,1)-q(:,2))/tau)); 461 | ppost = eps*0.5+(1-eps)*ppost; % to prevent log(0) 462 | 463 | 464 | end % end ggpost 465 | 466 | end 467 | 468 | 469 | 470 | 471 | 472 | 473 | 474 | 475 | -------------------------------------------------------------------------------- /SectionD/parameter_recovery/parameter_recovery_simulate.mlx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/SectionD/parameter_recovery/parameter_recovery_simulate.mlx -------------------------------------------------------------------------------- /SectionD/parameter_recovery/sampleFromArbitraryP.m: -------------------------------------------------------------------------------- 1 | function [X] = sampleFromArbitraryP(p,gridX,N) 2 | % inverse transform sampling scheme 3 | % function [X] = sampleFromArbitraryP(p,gridX,N) 4 | % This function samples from an arbitrary 1D probability distribution 5 | % IN: 6 | % - p: pX1 vector (the density evaluated along the grid) 7 | % - gridX: pX1 vector (the grid over which the density is evaluated) 8 | % - N: the number of samples to be sampled 9 | % OUT: 10 | % - X: NX1 vector of samples 11 | 12 | try; N; catch, N=1; end 13 | p = reshape(p,numel(p),1); 14 | 15 | 16 | if size(gridX,1)==1 17 | gridX = reshape(gridX,numel(gridX),1) 18 | 19 | end 20 | k = size(gridX,2); 21 | 22 | if any(isnan(p)) 23 | X = nan(N,k); 24 | else 25 | pcdf = cumsum(p(:)); 26 | X = zeros(N,k); 27 | for i=1:N 28 | below = find(rand<=pcdf); 29 | X(i,:) = gridX(below(1),:); 30 | end 31 | end -------------------------------------------------------------------------------- /TutorialMaterials/README.md: -------------------------------------------------------------------------------- 1 | This folder contains materials related to the afternoon tutorials. 2 | 3 | Tutorial 1: Inferring cognitive models of reinforcement learning from choice data 4 | See Tutorial1 folder 5 | 6 | Tutorial 2: Uncovering heterogeneity in preferences and behavior with finite mixture models 7 | See Tutorial2 folder 8 | 9 | Tutorial 4: Computational models of human gaze data 10 | Google Colab tutorial introduction by Angela Radulescu: https://colab.research.google.com/drive/1JN2HSPf7CD-9VRUhPK1itMtGDmqcbfX2#scrollTo=Baq0ayyXnS_N 11 | 12 | Tutorial 5: Computational modeling of goal-directed and habitual reinforcement-learning strategies 13 | GitHub repo with code and solutions by Claire Smid & Wouter Kool: https://github.com/ClaireSmid/Model-free_Model-based_Workshop 14 | 15 | -------------------------------------------------------------------------------- /TutorialMaterials/Tutorial1/README.md: -------------------------------------------------------------------------------- 1 | Tutorial 1: Inferring cognitive models of reinforcement learning from choice data 2 | 3 | Enclosed are the codes that will be used in Tutorial 2. They are matlab/octave compatible. 4 | 5 | Tutorial led by Maël Lebreton & Stefano Palminteri 6 | -------------------------------------------------------------------------------- /TutorialMaterials/Tutorial1/estimate_QnLL_gridsearch.m: -------------------------------------------------------------------------------- 1 | clear all 2 | close all 3 | clc 4 | 5 | 6 | %-------------------------------------------------------------------------% 7 | % SIMULATION OF THE SIMPLE Q-LEARNING MODEL FOR INSTRUMENTAL CONDITIONING 8 | % 9 | %-------------------------------------------------------------------------% 10 | 11 | 12 | load('sim_data_sess') 13 | 14 | %% -------------------------- INPUTS ------------------------------------- 15 | % NUMBER OF TRIALS 16 | ntrials = 24; 17 | 18 | % INITIAL VALUE 19 | Q0 = [0.5 0.5]; % initial value 20 | 21 | % PARAMETERS 22 | alpha_mat = linspace(0,1,201); 23 | temp_mat = linspace(0,5,201); 24 | 25 | nLL = NaN(numel(alpha_mat),numel(temp_mat)); 26 | 27 | for a = 1:length(alpha_mat) 28 | alpha = alpha_mat(a); 29 | 30 | for b = 1:length(temp_mat) 31 | inv_temp = temp_mat(b); 32 | 33 | 34 | PA = NaN(ntrials,nsess); 35 | lik = NaN(ntrials,nsess); 36 | Qt = NaN(ntrials,2,nsess); 37 | PE = NaN(ntrials,nsess); 38 | 39 | for ksess = 1:nsess 40 | 41 | %% ------------------------- Q SIMULATION -------------------------------- 42 | Qt(1,:,ksess) = Q0; 43 | 44 | % value simulation 45 | for t = 1:ntrials 46 | 47 | PA(t,ksess) = 1./(1+exp(-inv_temp.*(Qt(t,2,ksess)-Qt(t,1,ksess)))); 48 | 49 | if ch(t,ksess) == 1 50 | lik(t,ksess) = 1 - PA(t,ksess); 51 | elseif ch(t,ksess) == 2 52 | lik(t,ksess) = PA(t,ksess); 53 | end 54 | 55 | % compute prediction error 56 | PE(t,ksess) = r(t,ksess) - Qt(t,ch(t,ksess),ksess); 57 | 58 | % update value 59 | Qt(t+1,ch(t,ksess),ksess) = Qt(t,ch(t,ksess),ksess) + alpha.*PE(t,ksess); 60 | Qt(t+1,3-ch(t,ksess),ksess) = Qt(t,3-ch(t,ksess),ksess); 61 | 62 | end 63 | end 64 | 65 | nLL(a,b) = -sum(log(lik(:))); 66 | end 67 | end 68 | 69 | 70 | %% 71 | figure 72 | 73 | imagesc(flipud(nLL)) 74 | hold on 75 | xlabel('inverse temperature') 76 | ylabel('learning rate') 77 | 78 | xt = linspace(1,length(temp_mat),11); 79 | xtl = linspace(min(temp_mat),max(temp_mat),11); 80 | 81 | yt = linspace(1,length(alpha_mat),11); 82 | ytl = linspace(min(alpha_mat),max(alpha_mat),11); 83 | 84 | set(gca,'XLim',[1 length(temp_mat)],... 85 | 'XTick',xt,... 86 | 'XTickLabel',xtl,... 87 | 'YLim',[1 length(alpha_mat)],... 88 | 'YTick',yt,... 89 | 'YTickLabel',fliplr(ytl)) 90 | 91 | 92 | colorbar 93 | 94 | 95 | [I,J] = find(nLL == min(min(nLL))); 96 | 97 | disp(strcat('alpha = ',num2str(alpha_mat(I)))) 98 | disp(strcat('inv. temp = ',num2str(temp_mat(J)))) 99 | 100 | 101 | plot(J,length(alpha_mat)-I,'o',... 102 | 'MarkerFaceColor',[1,0,0],... 103 | 'MarkerEdgeColor',[1,0,0]) 104 | 105 | text(20,35,strcat('true \alpha = ',num2str(alphaSim)),'Fontsize',14,'color',[1, 1, 1]) 106 | text(20,25,strcat('true \beta = ',num2str(betaSim)),'Fontsize',14,'color',[1, 1, 1]) 107 | 108 | 109 | -------------------------------------------------------------------------------- /TutorialMaterials/Tutorial1/fullrecovery_QnLL_fmincon.m: -------------------------------------------------------------------------------- 1 | clear all 2 | close all 3 | clc 4 | 5 | 6 | %-------------------------------------------------------------------------% 7 | % SIMULATION OF THE SIMPLE Q-LEARNING MODEL FOR INSTRUMENTAL CONDITIONING 8 | % 9 | %-------------------------------------------------------------------------% 10 | 11 | nfl = dir('sim_data_sub*'); 12 | nsub = length(nfl); 13 | 14 | estimP = NaN(nsub,2); 15 | simuP = NaN(nsub,2); 16 | 17 | for ksub = 1:nsub 18 | 19 | load(strcat('sim_data_sub',num2str(ksub))) 20 | 21 | %% -------------------------- INPUTS ------------------------------------- 22 | 23 | x0 = [5 0.5]; 24 | xmin = [0 0]; 25 | xmax = [50 1]; 26 | 27 | options = optimset('Algorithm', 'interior-point', 'Display', 'iter-detailed', 'MaxIter', 10000); % These increase the number of iterations to ensure the convergence 28 | 29 | [parameters,nll,~,~,~] = fmincon(@(x) Qlearner(x,ch,r),x0,[],[],[],[],xmin,xmax,[],options); 30 | 31 | 32 | estimP(ksub,:) = parameters; 33 | simuP(ksub,:) = [betaSim,alphaSim]; 34 | 35 | 36 | end 37 | 38 | 39 | figure 40 | subplot(1,2,1) 41 | hold on 42 | plot([0,10],[0,10],'--k') 43 | [b,stats] = robustfit(simuP(:,1),estimP(:,1)); 44 | XX = linspace(min(simuP(:,1)),max(simuP(:,1)),1000); 45 | [Yf] = glmval(b,XX,'identity'); 46 | XXX = sortrows([XX',Yf],1); 47 | hModel = plot(XXX(:,1),XXX(:,2),'-',... 48 | 'Color',.85*[0,0,1],... 49 | 'LineWidth',2); 50 | 51 | plot(simuP(:,1),estimP(:,1),'o',... 52 | 'MarkerFaceColor',[1,1,1],... 53 | 'MarkerEdgeColor',[0,0,0]) 54 | xlabel('true \beta') 55 | ylabel('recovered \beta') 56 | 57 | subplot(1,2,2) 58 | hold on 59 | plot([0,1],[0,1],'--k') 60 | [b,stats] = robustfit(simuP(:,2),estimP(:,2)); 61 | XX = linspace(min(simuP(:,2)),max(simuP(:,2)),1000); 62 | [Yf] = glmval(b,XX,'identity'); 63 | XXX = sortrows([XX',Yf],1); 64 | hModel = plot(XXX(:,1),XXX(:,2),'-',... 65 | 'Color',.85*[0,0,1],... 66 | 'LineWidth',2); 67 | plot(simuP(:,2),estimP(:,2),'o',... 68 | 'MarkerFaceColor',[1,1,1],... 69 | 'MarkerEdgeColor',[0,0,0]) 70 | xlabel('true \alpha') 71 | ylabel('recovered \alpha') 72 | 73 | %% 74 | %%========================================================== 75 | 76 | 77 | function nLL = Qlearner(params,choice,reward) 78 | 79 | inv_temp = params(1); 80 | alpha = params(2); 81 | 82 | nsess = size(choice,2); 83 | ntrials = size(choice,1); 84 | 85 | PA = NaN(ntrials,nsess); 86 | lik = NaN(ntrials,nsess); 87 | Qt = NaN(ntrials,2,nsess); 88 | PE = NaN(ntrials,nsess); 89 | 90 | % INITIAL VALUE 91 | Q0 = [0.5 0.5]; % initial value 92 | 93 | for ksess = 1:nsess 94 | 95 | %% ------------------------- Q SIMULATION -------------------------------- 96 | Qt(1,:,ksess) = Q0; 97 | 98 | % value simulation 99 | for t = 1:ntrials 100 | 101 | PA(t,ksess) = 1./(1+exp(-inv_temp.*(Qt(t,2,ksess)-Qt(t,1,ksess)))); 102 | 103 | if choice(t,ksess) == 1 104 | lik(t,ksess) = 1 - PA(t,ksess); 105 | elseif choice(t,ksess) == 2 106 | lik(t,ksess) = PA(t,ksess); 107 | end 108 | 109 | % compute prediction error 110 | PE(t,ksess) = reward(t,ksess) - Qt(t,choice(t,ksess),ksess); 111 | 112 | % update value 113 | Qt(t+1,choice(t,ksess),ksess) = Qt(t,choice(t,ksess),ksess) + alpha.*PE(t,ksess); 114 | Qt(t+1,3-choice(t,ksess),ksess) = Qt(t,3-choice(t,ksess),ksess); 115 | 116 | end 117 | end 118 | nLL = -sum(log(lik(:))); 119 | 120 | end -------------------------------------------------------------------------------- /TutorialMaterials/Tutorial1/simulateQ_one.m: -------------------------------------------------------------------------------- 1 | %-------------------------------------------------------------------------% 2 | % SIMULATION OF THE SIMPLE Q-LEARNING MODEL FOR INSTRUMENTAL CONDITIONING 3 | % 4 | %-------------------------------------------------------------------------% 5 | 6 | 7 | %% -------------------------- INPUTS ------------------------------------- 8 | 9 | % PARAMETERS 10 | alpha = 0.3; 11 | inv_temp = 1.5; 12 | 13 | % NUMBER OF TRIALS 14 | ntrials = 24; 15 | 16 | 17 | %% ------------------------- PARAMETERS ----------------------------------- 18 | 19 | % INITIAL VALUE 20 | Q0 = [0.5 0.5]; % initial value 21 | 22 | % REWARD HISTORY 23 | RA = rand(ntrials,1)<0.8; 24 | RB = rand(ntrials,1)<0.2; 25 | O = [RB, RA]; 26 | 27 | 28 | %% ------------------------- Q SIMULATION -------------------------------- 29 | 30 | % initialise value and prediction error vectors to store the data 31 | Qt = nan(ntrials+1, 2); % store the Q-values for option B (col 1) and option A (col 2) 32 | PE = nan(ntrials, 1); % store the Prediction error 33 | ch = nan(ntrials, 1); % store the choice : 1 = B; 2 = A 34 | PA = nan(ntrials, 1); % store the modelled proba. of choosing A 35 | r = nan(ntrials, 1); % store the obtained reward 36 | 37 | Qt(1,:) = Q0; % initalise Q values 38 | 39 | % value simulation 40 | for t = 1:ntrials 41 | 42 | PA(t) = 1./(1+exp(-inv_temp.*(Qt(t,2)-Qt(t,1)))); 43 | ch(t) = 1 + double(rand()= 0 && isfield(params,'alpha_pos') % separate learning rates 30 | alpha = params.alpha_pos; 31 | elseif PE < 0 && isfield(params,'alpha_neg') % separate learning rates 32 | alpha = params.alpha_neg; 33 | else % only one learning rate 34 | alpha = params.alpha; 35 | end 36 | 37 | %% update value 38 | V = V + alpha * PE; -------------------------------------------------------------------------------- /general_functions/softmax.m: -------------------------------------------------------------------------------- 1 | % function [policy] = softmax(Vs,params) 2 | % 3 | % Softmax function to transform choice values into choice policy 4 | % 5 | % input: 6 | % @Vs: array of choice values for all available options at trial t 7 | % @params.tau: decision temperature parameter (cave: the larger, the more random - inverse of the 'inverse temperature') 8 | % 9 | % output: 10 | % @policy: array of choice probabilities for each available option 11 | % 12 | % Tobias Hauser, 06/2021 13 | % 14 | function [policy] = softmax(Vs,params) 15 | 16 | %% get parameter 17 | tau = params.tau; 18 | 19 | 20 | %% calculate policy 21 | 22 | % remove max to avoid numerical overflow (cf Friston spm_softmax) 23 | % already divide by temperature tau 24 | VT = (Vs - max(Vs)) / tau; 25 | 26 | % softmax transformation 27 | policy = (exp(VT) / sum(exp(VT))); -------------------------------------------------------------------------------- /slides_morning/2021_09_17_Welcome_FLUX_workshop.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/slides_morning/2021_09_17_Welcome_FLUX_workshop.pdf -------------------------------------------------------------------------------- /slides_morning/FLUXMorningSession_Introduction.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/slides_morning/FLUXMorningSession_Introduction.pdf -------------------------------------------------------------------------------- /slides_morning/FLUXSectionB_How to develop a computational model.pptx.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DevComPsy/2021FluxCompModellingWorkshop/3c1019026fad2181331cf6238f1e2705ac980bd6/slides_morning/FLUXSectionB_How to develop a computational model.pptx.pdf --------------------------------------------------------------------------------