├── .README.md.swp ├── LICENSE ├── README.md ├── SMC.py ├── ex_7dimgaussian.py ├── ex_bivariategaussian.py └── figures ├── Figure_1.png └── Figure_2.png /.README.md.swp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rishabhdutta/SMC-python/052c26b90278e13548fe6f408d38432f5f193c3f/.README.md.swp -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Rishabh Dutta 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # SMC-python 2 | Python codes for Sequential Monte Carlo sampling Technique. 3 | 4 | This technique is robust in sampling close to 1000-dimensions posterior probability 5 | densities. 6 | 7 | Uses `numpy`, `scipy` and `collections` libraries. 8 | 9 | ## Demonstration 10 | ### Example 1: Bivariate bimodal Gaussian probability density 11 | 12 | We draw samples from a bivariate bimodal Gaussian probability density as shown below: 13 | ![Image of Yaktocat](https://github.com/rishabhdutta/SMC-python/blob/master/figures/Figure_1.png) 14 | The two modes for first parameter (`par 1`) is at `[2 10]`, while for the second 15 | parameter (`par 2`) is at `[6 12]`. The corresponding standard deviation are 16 | `[2 1]` for `par 1` and `[2 1]` for `par 2`, respectively. 17 | 18 | #### Codes to calculate target density for the two parameters 19 | `postt` calculates the log of the target distribution. 20 | ``` 21 | import numpy as np 22 | from scipy.stats import multivariate_normal 23 | 24 | mu1 = np.array([2,12]) 25 | sigma1 = np.array([2,1]) 26 | mvn1 = multivariate_normal(mu1,sigma1) 27 | 28 | mu2 = np.array([10,6]) 29 | sigma2 = np.array([1,2]) 30 | mvn2 = multivariate_normal(mu2,sigma2) 31 | 32 | postt = lambda x: np.log((mvn1.pdf(x)+mvn2.pdf(x))/ \ 33 | (mvn1.pdf(mu1)+mvn2.pdf(mu2))) 34 | ``` 35 | 36 | #### Generate samples corresponding to the target distribution 37 | Import the `collections` library. 38 | ``` 39 | from collections import namedtuple 40 | ``` 41 | Create two named tuple objects: `opt` and `samples`. 42 | 43 | `opt` corresponds to input parameters of the sampling technique. `opt.N` is the 44 | number of Markov chains, `opt.Neff` is the chain length, `opt.LB` is the lower 45 | bound of the target distribution parameters, and `opt.UB` is the upper bound. 46 | 47 | `samples` corresponds to the information of the samples at each intermediate 48 | stage. `samples.allsamples` is the ensemble of samples at final stage, `samples.postval` 49 | is the log of target distribution values, `samples.stage` is the array of all 50 | the stages, `samples.beta` is the array of beta parameters, `samples.covsmpl` is 51 | the sample covariance at final stage, `samples.resmpl` is the resampled samples 52 | at the final stage. 53 | ``` 54 | NT1 = namedtuple('NT1', 'N Neff target LB UB') 55 | opt = NT1(2000, 50, postt, np.array([0,0]), np.array([15,15])) 56 | 57 | NT2 = namedtuple('NT2', 'allsamples postval beta stage covsmpl resmpl') 58 | samples = NT2(None, None, None, None, None, None) 59 | ``` 60 | Run this line to get all the samples. 61 | ``` 62 | final = SMC_samples(opt,samples, NT1, NT2) 63 | ``` 64 | #### Plot the histograms of the resulting samples 65 | ``` 66 | plt.subplot(1, 2, 1) 67 | n, bins, patches = plt.hist(final.allsamples[:,0], 50, density=True, \ 68 | facecolor='b', alpha=0.75) 69 | plt.subplot(1, 2, 2) 70 | n, bins, patches = plt.hist(final.allsamples[:,1], 50, density=True, \ 71 | facecolor='b', alpha=0.75) 72 | ``` 73 | ![Image2](https://github.com/rishabhdutta/SMC-python/blob/master/figures/Figure_2.png) 74 | 75 | ### Citation 76 | 77 | If you use our toolbox or if you find our research helpful, please cite the following paper (thanks for your support): 78 | 79 | Dutta, R., Jónsson, S., & Vasyua-Bathke, H. (2021). Simultaneous Bayesian estimation of non-planar fault geometry and spatially-variable slip. Journal of Geophysical Research: Solid Earth, 126(7), e2020JB020441. 80 | 81 | Any feedback is welcome! (rishabh421989@gmail.com) 82 | 83 | 84 | -------------------------------------------------------------------------------- /SMC.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Tue Mar 26 11:00:48 2019 5 | 6 | @author: duttar 7 | """ 8 | import numpy as np 9 | 10 | def deterministicR(inIndex,q): 11 | """ 12 | Parameters: 13 | * inIndex : index of the posterior values 14 | * q : weight of the posterior values 15 | 16 | Output: 17 | * outindx : index of the resampling 18 | """ 19 | 20 | n_chains = inIndex.shape[0] 21 | parents = np.arange(n_chains) 22 | N_childs = np.zeros(n_chains, dtype=int) 23 | 24 | cum_dist = np.cumsum(q) 25 | aux = np.random.rand(1) 26 | u = parents + aux 27 | u /= n_chains 28 | j = 0 29 | for i in parents: 30 | while u[i] > cum_dist[j]: 31 | j += 1 32 | N_childs[j] += 1 33 | 34 | indx = 0 35 | outindx = np.zeros(n_chains, dtype=int) 36 | for i in parents: 37 | if N_childs[i] > 0: 38 | for j in range(indx, (indx + N_childs[i])): 39 | outindx[j] = parents[i] 40 | 41 | indx += N_childs[i] 42 | 43 | return outindx 44 | 45 | def AMH(X,target,covariance,mrun,beta,LB,UB): 46 | """ 47 | Adaptive Metropolis algorithm 48 | scales the covariance matrix according to the acceptance rate 49 | cov of proposal = (a+ bR)*sigma; R = acceptance rate 50 | returns the last sample of the chain 51 | 52 | Parameters: 53 | * X : starting model 54 | * target : target distribution (is a function handle and calculates the log posterior) 55 | * covariance : covariance of the proposal distribution 56 | * mrun : number of samples 57 | * beta : use the beta parameter of SMC sampling, otherwise 1 58 | * LB : lower bound of the model parameters 59 | * UB : upper bound of the model parameters 60 | 61 | Outputs: 62 | * G : last sample of the chain 63 | * GP : log posterior value of the last sample 64 | * avg_acc : average acceptance rate 65 | 66 | written by : Rishabh Dutta (18 Mar 2019) 67 | Matlab version written on 12 Mar 2016 68 | (Don't forget to acknowledge) 69 | """ 70 | 71 | Dims = covariance.shape[0] 72 | logpdf = target(X) 73 | V = covariance 74 | best_P = logpdf * beta 75 | P0 = logpdf * beta 76 | 77 | # the following values are estimated empirically 78 | a = 1/9 79 | b = 8/9 80 | 81 | sameind = np.where(np.equal(LB, UB)) 82 | 83 | def dimension(i): 84 | switcher={ 85 | 1:0.441, 86 | 2:0.352, 87 | 3:0.316, 88 | 4:0.285, 89 | 5:0.275, 90 | 6:0.273, 91 | 7:0.270, 92 | 8:0.268, 93 | 9:0.267, 94 | 10:0.266, 95 | 11:0.265, 96 | 12:0.255 97 | } 98 | return switcher.get(i, 0.255) 99 | 100 | # set initial scaling factor 101 | s = a + b*dimension(Dims) 102 | 103 | U = np.log(np.random.rand(1,mrun)) 104 | TH = np.zeros((Dims,mrun)) 105 | THP = np.zeros((1,mrun)) 106 | avg_acc = 0 107 | factor = np.zeros((1,mrun)) 108 | 109 | for i in range(mrun): 110 | X_new = np.random.multivariate_normal(X,s**2*V) 111 | X_new[sameind] = LB[sameind] 112 | 113 | ind1 = np.where(X_new < LB) 114 | diff1 = LB[ind1] - X_new[ind1] 115 | X_new[ind1] = LB[ind1] + diff1 116 | 117 | if avg_acc < 0.05: 118 | X_new[ind1] = LB[ind1] 119 | 120 | ind2 = np.where(X_new > UB) 121 | diff2 = X_new[ind2] - UB[ind2] 122 | X_new[ind2] = UB[ind2] - diff2 123 | 124 | if avg_acc < 0.05: 125 | X_new[ind2] = UB[ind2] 126 | 127 | P_new = beta * target(X_new) 128 | 129 | if P_new > best_P: 130 | X = X_new 131 | best_P = P_new 132 | P0 = P_new 133 | acc_rate = 1 134 | else: 135 | rho = P_new - P0 136 | acc_rate = np.exp(np.min([0,rho])) 137 | if U[0,i] <= rho : 138 | X = X_new 139 | P0 = P_new 140 | 141 | TH[:,i] = np.transpose(X) 142 | THP[0,i] = P0 143 | factor[0,i] = s**2 144 | avg_acc = avg_acc*(i)/(i+1) + acc_rate/(i+1) 145 | s = a+ b*avg_acc 146 | 147 | G = TH[:,-1] 148 | GP = THP[0,-1]/beta 149 | 150 | return G, GP, avg_acc 151 | 152 | # %% 153 | class SMCclass: 154 | """ 155 | Generates samples of the 'target' posterior PDF using SMC sampling. Also called Adapative Transitional Metropolis 156 | Importance (sampling) P abbreviated as ATMIP 157 | """ 158 | def __init__(self, opt, samples, NT1, NT2, verbose=True): 159 | """ 160 | Parameters: 161 | opt : named tuple 162 | - opt.target (lamda function of the posterior) 163 | - opt.UB (upper bound of parameters) 164 | - opt.LB (lower bound of parameters) 165 | - opt.N (number of Markov chains at each stage) 166 | - opt.Neff (Chain length of the MCMC sampling) 167 | 168 | samples: named tuple 169 | - samples.allsamples (samples at each stage) 170 | - samples.postval (log posterior value of samples) 171 | - samples.beta (array of beta values) 172 | - samples.stage (array of stages) 173 | - samples.covsmpl (model covariance at each stage) 174 | - samples.resmpl (resampled model at each stage) 175 | 176 | NT1: create opt object 177 | NT2: create samples object 178 | 179 | written by: Rishabh Dutta, Dec 12 2018 180 | (Don't forget to acknowledge) 181 | 182 | """ 183 | self.verbose = verbose 184 | self.opt = opt 185 | self.samples = samples 186 | self.NT1 = NT1 187 | self.NT2 = NT2 188 | 189 | def initialize(self): 190 | if self.verbose: 191 | print ("-----------------------------------------------------------------------------------------------") 192 | print ("-----------------------------------------------------------------------------------------------") 193 | print(f'Initializing ATMIP with {self.opt.N :8d} Markov chains and {self.opt.Neff :8d} chain length.') 194 | 195 | def prior_samples(self): 196 | ''' 197 | determines the prior posterior values 198 | the prior samples are estimated from lower and upper bounds 199 | 200 | Output : samples (NT2 object with estimated posterior values) 201 | ''' 202 | numpars = self.opt.LB.shape[0] 203 | diffbnd = self.opt.UB - self.opt.LB 204 | diffbndN = np.tile(diffbnd,(self.opt.N,1)) 205 | LBN = np.tile(self.opt.LB,(self.opt.N,1)) 206 | 207 | sampzero = LBN + np.random.rand(self.opt.N,numpars) * diffbndN 208 | beta = np.array([0]) 209 | stage = np.array([1]) 210 | 211 | postval = np.zeros([self.opt.N,1]) 212 | for i in range(self.opt.N): 213 | samp0 = sampzero[i,:] 214 | logpost = self.opt.target(samp0) 215 | postval[i] = logpost 216 | 217 | samples = self.NT2(sampzero, postval, beta, stage, None, None) 218 | return samples 219 | 220 | 221 | def find_beta(self): 222 | """ 223 | Calculates the beta parameter for the next stage 224 | """ 225 | beta1 = self.samples.beta[-1] #prev_beta 226 | beta2 = self.samples.beta[-1] #prev_beta 227 | max_post = np.max(self.samples.postval) 228 | logpst = self.samples.postval - max_post 229 | beta = beta1+.5 230 | 231 | if beta>1: 232 | beta = 1 233 | #logwght = beta.*logpst 234 | #wght = np.exp(logwght) 235 | 236 | refcov = 1 237 | 238 | while beta - beta1 > 1e-6: 239 | curr_beta = (beta+beta1)/2 240 | diffbeta = beta-beta1 241 | logwght = diffbeta*logpst 242 | wght = np.exp(logwght) 243 | covwght = np.std(wght)/np.mean(wght) 244 | if covwght > refcov: 245 | beta = curr_beta 246 | else: 247 | beta1 = curr_beta 248 | 249 | betanew = np.min(np.array([1,beta])) 250 | betaarray = np.append(self.samples.beta,betanew) 251 | newstage = np.arange(1,self.samples.stage[-1]+2) 252 | samples = self.NT2(self.samples.allsamples, self.samples.postval, \ 253 | betaarray, newstage, self.samples.covsmpl, \ 254 | self.samples.resmpl) 255 | 256 | return samples 257 | 258 | def resample_stage(self): 259 | ''' 260 | Resamples the model samples at a certain stage 261 | Uses Kitagawa's deterministic resampling algorithm 262 | ''' 263 | 264 | # calculate the weight for model samples 265 | logpst = self.samples.postval - np.max(self.samples.postval) 266 | logwght = (self.samples.beta[-1] - self.samples.beta[-2])* logpst 267 | wght = np.exp(logwght) 268 | 269 | probwght = wght/np.sum(wght) 270 | inind = np.arange(0,self.opt.N) 271 | 272 | outind = deterministicR(inind, probwght) 273 | newsmpl = self.samples.allsamples[outind,:] 274 | 275 | samples = self.NT2(self.samples.allsamples, self.samples.postval, \ 276 | self.samples.beta, self.samples.stage, \ 277 | self.samples.covsmpl, newsmpl) 278 | 279 | return samples 280 | 281 | def make_covariance(self): 282 | ''' 283 | make the model covariance using the weights and samples from previous 284 | stage 285 | ''' 286 | # calculate the weight for model samples 287 | 288 | dims = self.samples.allsamples.shape[1] 289 | logpst = self.samples.postval - np.max(self.samples.postval) 290 | logwght = (self.samples.beta[-1] - self.samples.beta[-2])* logpst 291 | wght = np.exp(logwght) 292 | 293 | probwght = wght/np.sum(wght) 294 | weightmat = np.tile(probwght,(1,dims)) 295 | multmat = weightmat * self.samples.allsamples 296 | 297 | # calculate the mean samples 298 | meansmpl = multmat.sum(axis=0, dtype='float') 299 | 300 | # calculate the model covariance 301 | covariance = np.matrix(np.zeros((dims,dims), dtype='float')) 302 | for i in range(self.opt.N): 303 | par = self.samples.allsamples[i,:] 304 | smpldiff = np.matrix(par - meansmpl) 305 | smpdsq = np.matmul(np.transpose(smpldiff),smpldiff) 306 | covint = np.multiply(probwght[i], smpdsq) 307 | covariance += covint 308 | 309 | samples = self.NT2(self.samples.allsamples, self.samples.postval, \ 310 | self.samples.beta, self.samples.stage, \ 311 | covariance, self.samples.resmpl) 312 | return samples 313 | 314 | def MCMC_samples(self): 315 | """ 316 | Nothing 317 | """ 318 | dims = self.samples.allsamples.shape[1] 319 | 320 | mhsmpl = np.zeros([self.opt.N,dims]) 321 | mhpost = np.zeros([self.opt.N,1]) 322 | for i in range(self.opt.N): 323 | start = self.samples.resmpl[i,:] 324 | G, GP, acc = AMH(start, self.opt.target, self.samples.covsmpl, \ 325 | self.opt.Neff, self.samples.beta[-1], \ 326 | self.opt.LB, self.opt.UB) 327 | mhsmpl[i,:] = np.transpose(G) 328 | mhpost[i] = GP 329 | 330 | samples = self.NT2(mhsmpl, mhpost, self.samples.beta, \ 331 | self.samples.stage, self.samples.covsmpl, \ 332 | self.samples.resmpl) 333 | return samples 334 | 335 | 336 | # %% 337 | def SMC_samples(opt,samples, NT1, NT2): 338 | ''' 339 | Sequential Monte Carlo technique 340 | < a subset of CATMIP by Sarah Minson> 341 | The method samples the target distribution through several stages (called 342 | transitioning of simulated annealing). At each stage the samples corresponds 343 | to the intermediate PDF between the prior PDF and final target PDF. 344 | 345 | After samples generated at each stage, the beta parameter is generated for 346 | the next stage. At the next stage, resampling is performed. Then MCMC 347 | sampling (adpative Metropolis chains) is resumed from each resampled model. 348 | The weigted covariance is estimated using the weights (calculated from 349 | posterior values) and samples from previous stage. This procedure is conti- 350 | nued until beta parameter is 1. 351 | 352 | syntax: output = ATMIP(opt) 353 | 354 | Inputs: 355 | 356 | opt : named tuple 357 | - opt.target (lamda function of the posterior) 358 | - opt.UB (upper bound of parameters) 359 | - opt.LB (lower bound of parameters) 360 | - opt.N (number of Markov chains at each stage) 361 | - opt.Neff (Chain length of the MCMC sampling) 362 | 363 | samples: named tuple 364 | - samples.allsamples (samples at an intermediate stage) 365 | - samples.beta (beta at that stage) 366 | - samples.postval (posterior values) 367 | - samples.stage (stage number) 368 | - samples.covsmpl (model covariance matrix used for MCMC sampling) 369 | 370 | NT1 - named tuple structure for opt 371 | NT2 - named tuple structure for samples 372 | 373 | Outputs: 374 | 375 | samples : named tuple 376 | - samples.allsamples (final samples at the last stage) 377 | - samples.postval (log posterior values of the final samples) 378 | - samples.stages (array of all stages) 379 | - samples.beta (array of beta values) 380 | - samples.covsmpl (model covariance at final stage) 381 | - samples.resmpl (resampled model samples at final stage) 382 | 383 | written by: Rishabh Dutta, Mar 25 2019 384 | (Don't forget to acknowledge) 385 | ''' 386 | current = SMCclass(opt, samples, NT1, NT2) 387 | current.initialize() # prints the initialization 388 | 389 | if samples.allsamples is None: # generates prior samples and calculates 390 | # their posterior values 391 | print('------Calculating the prior posterior values at stage 1-----') 392 | current = SMCclass(opt, samples, NT1, NT2) 393 | samples = current.prior_samples() 394 | 395 | while samples.beta[-1] != 1: 396 | current = SMCclass(opt, samples, NT1, NT2) 397 | samples = current.find_beta() # calculates beta at next stage 398 | 399 | # at next stage here ------------------------------------- 400 | current = SMCclass(opt, samples, NT1, NT2) 401 | samples = current.resample_stage() # resample the model samples 402 | 403 | # make the model covariance 404 | current = SMCclass(opt, samples, NT1, NT2) 405 | samples = current.make_covariance() 406 | 407 | # use the resampled model samples as starting point for MCMC sampling 408 | # we use adaptive Metropolis sampling 409 | # adaptive proposal is generated using model covariance 410 | print(f'Starting metropolis chains at stage = {samples.stage[-1] :3d} and beta = {samples.beta[-1] :.6f}.') 411 | current = SMCclass(opt, samples, NT1, NT2) 412 | samples = current.MCMC_samples() 413 | 414 | return samples 415 | -------------------------------------------------------------------------------- /ex_7dimgaussian.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Wed Mar 27 17:35:03 2019 5 | 6 | @author: duttar 7 | """ 8 | import numpy as np 9 | from scipy.stats import multivariate_normal 10 | from mpl_toolkits.mplot3d import Axes3D 11 | import matplotlib.pyplot as plt 12 | from matplotlib import cm 13 | from matplotlib.ticker import LinearLocator, FormatStrFormatter 14 | 15 | # create a bimodal bivariate normal distribution 16 | 17 | mu1 = np.array([2,12,3,12,4,11,3]) 18 | sigma1 = np.array([2,1,2,1,1,2,1]) 19 | mvn1 = multivariate_normal(mu1,sigma1) 20 | 21 | mu2 = np.array([10,6,12,4,11,2,10]) 22 | sigma2 = np.array([1,2,1,2,2,1,2]) 23 | mvn2 = multivariate_normal(mu2,sigma2) 24 | 25 | postt = lambda x: np.log((mvn1.pdf(x)+mvn2.pdf(x))/ \ 26 | (mvn1.pdf(mu1)+mvn2.pdf(mu2))) 27 | 28 | #%% 29 | from collections import namedtuple 30 | import matplotlib.pyplot as plt 31 | runfile('SMC.py', wdir='.') 32 | 33 | # create tuple object with desired number of markov chains, chain length and 34 | # lower bound and upper bound of the model 35 | NT1 = namedtuple('NT1', 'N Neff target LB UB') 36 | opt = NT1(2000, 50, postt, np.array([0,0,0,0,0,0,0]), \ 37 | np.array([15,15,15,15,15,15,15])) 38 | 39 | # tuple object for the samples 40 | NT2 = namedtuple('NT2', 'allsamples postval beta stage covsmpl resmpl') 41 | samples = NT2(None, None, None, None, None, None) 42 | 43 | # run the SMC sampling 44 | final = SMC_samples(opt,samples, NT1, NT2) 45 | 46 | # plot the results: histograms 47 | plt.subplot(2, 4, 1) 48 | n, bins, patches = plt.hist(final.allsamples[:,0], 50, density=True, \ 49 | facecolor='b', alpha=0.75) 50 | plt.xlabel('Par 1') 51 | plt.ylabel('Probability') 52 | plt.title('Histogram') 53 | plt.grid(True) 54 | 55 | plt.subplot(2, 4, 2) 56 | n, bins, patches = plt.hist(final.allsamples[:,1], 50, density=True, \ 57 | facecolor='b', alpha=0.75) 58 | plt.xlabel('Par 2') 59 | plt.ylabel('Probability') 60 | plt.title('Histogram') 61 | plt.grid(True) 62 | 63 | plt.subplot(2, 4, 3) 64 | n, bins, patches = plt.hist(final.allsamples[:,2], 50, density=True, \ 65 | facecolor='b', alpha=0.75) 66 | plt.xlabel('Par 3') 67 | plt.ylabel('Probability') 68 | plt.title('Histogram') 69 | plt.grid(True) 70 | 71 | plt.subplot(2, 4, 4) 72 | n, bins, patches = plt.hist(final.allsamples[:,3], 50, density=True, \ 73 | facecolor='b', alpha=0.75) 74 | plt.xlabel('Par 4') 75 | plt.ylabel('Probability') 76 | plt.title('Histogram') 77 | plt.grid(True) 78 | 79 | plt.subplot(2, 4, 5) 80 | n, bins, patches = plt.hist(final.allsamples[:,4], 50, density=True, \ 81 | facecolor='b', alpha=0.75) 82 | plt.xlabel('Par 5') 83 | plt.ylabel('Probability') 84 | plt.title('Histogram') 85 | plt.grid(True) 86 | 87 | plt.subplot(2, 4, 6) 88 | n, bins, patches = plt.hist(final.allsamples[:,5], 50, density=True, \ 89 | facecolor='b', alpha=0.75) 90 | plt.xlabel('Par 6') 91 | plt.ylabel('Probability') 92 | plt.title('Histogram') 93 | plt.grid(True) 94 | 95 | plt.subplot(2, 4, 7) 96 | n, bins, patches = plt.hist(final.allsamples[:,6], 50, density=True, \ 97 | facecolor='b', alpha=0.75) 98 | plt.xlabel('Par 7') 99 | plt.ylabel('Probability') 100 | plt.title('Histogram') 101 | plt.grid(True) 102 | plt.show() -------------------------------------------------------------------------------- /ex_bivariategaussian.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Created on Mon Mar 25 15:20:09 2019 5 | 6 | @author: duttar 7 | """ 8 | 9 | import numpy as np 10 | from scipy.stats import multivariate_normal 11 | from mpl_toolkits.mplot3d import Axes3D 12 | import matplotlib.pyplot as plt 13 | from matplotlib import cm 14 | from matplotlib.ticker import LinearLocator, FormatStrFormatter 15 | 16 | # create a bimodal bivariate normal distribution 17 | 18 | mu1 = np.array([2,12]) 19 | sigma1 = np.array([2,1]) 20 | mvn1 = multivariate_normal(mu1,sigma1) 21 | 22 | mu2 = np.array([10,6]) 23 | sigma2 = np.array([1,2]) 24 | mvn2 = multivariate_normal(mu2,sigma2) 25 | 26 | postt = lambda x: np.log((mvn1.pdf(x)+mvn2.pdf(x))/ \ 27 | (mvn1.pdf(mu1)+mvn2.pdf(mu2))) 28 | 29 | ''' 30 | plot the posterior 31 | ''' 32 | fig = plt.figure() 33 | ax = fig.gca(projection='3d') 34 | 35 | # Make data. 36 | X = np.arange(0, 15.2, 0.2) 37 | Y = np.arange(15.2, 0, -0.2) 38 | XX, YY = np.meshgrid(X, Y) 39 | R = np.sqrt(X**2 + Y**2) 40 | Z = np.zeros([XX.shape[0],XX.shape[1]]) 41 | for i in range(Z.shape[0]): 42 | for j in range(Z.shape[1]): 43 | Z[i,j] = postt(np.array([XX[i,j],YY[i,j]])) 44 | ZZ = np.exp(Z) 45 | 46 | # Plot the surface. 47 | surf = ax.plot_surface(XX, YY, ZZ, cmap=cm.coolwarm, 48 | linewidth=0, antialiased=False) 49 | 50 | # Customize the z axis. 51 | ax.set_zlim(-.1, 0.6) 52 | ax.zaxis.set_major_locator(LinearLocator(10)) 53 | ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f')) 54 | plt.xlabel('Par 1') 55 | plt.ylabel('Par 2') 56 | 57 | # Add a color bar which maps values to colors. 58 | fig.colorbar(surf, shrink=0.5, aspect=5) 59 | 60 | plt.show() 61 | 62 | # %% 63 | from collections import namedtuple 64 | import matplotlib.pyplot as plt 65 | runfile('SMC.py', wdir='.') 66 | 67 | # create tuple object with desired number of markov chains, chain length and 68 | # lower bound and upper bound of the model 69 | NT1 = namedtuple('NT1', 'N Neff target LB UB') 70 | opt = NT1(2000, 50, postt, np.array([0,0]), np.array([15,15])) 71 | 72 | # tuple object for the samples 73 | NT2 = namedtuple('NT2', 'allsamples postval beta stage covsmpl resmpl') 74 | samples = NT2(None, None, None, None, None, None) 75 | 76 | # run the SMC sampling 77 | final = SMC_samples(opt,samples, NT1, NT2) 78 | 79 | # plot the results: histograms 80 | plt.subplot(2, 1, 1) 81 | n, bins, patches = plt.hist(final.allsamples[:,0], 50, density=True, \ 82 | facecolor='b', alpha=0.75) 83 | 84 | plt.xlabel('Par 1') 85 | plt.ylabel('Probability') 86 | plt.grid(True) 87 | 88 | plt.subplot(2, 1, 2) 89 | n, bins, patches = plt.hist(final.allsamples[:,1], 50, density=True, \ 90 | facecolor='b', alpha=0.75) 91 | 92 | 93 | plt.xlabel('Par 2') 94 | plt.ylabel('Probability') 95 | plt.grid(True) 96 | plt.show() 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | -------------------------------------------------------------------------------- /figures/Figure_1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rishabhdutta/SMC-python/052c26b90278e13548fe6f408d38432f5f193c3f/figures/Figure_1.png -------------------------------------------------------------------------------- /figures/Figure_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rishabhdutta/SMC-python/052c26b90278e13548fe6f408d38432f5f193c3f/figures/Figure_2.png --------------------------------------------------------------------------------