├── .gitignore ├── CARstan.Rproj ├── Dockerfile ├── LICENSE ├── README.Rmd ├── README.md ├── README_files └── figure-html │ ├── compare-parameter-estimates-1.png │ ├── fit-prec-model-1.png │ ├── fit-sparse-model-1.png │ ├── make-scotland-map-1.png │ └── unnamed-chunk-1-1.png ├── blogpost.md ├── data ├── scotland.dbf ├── scotland.shp ├── scotland.shx └── scotland_lip_cancer.RData └── stan ├── .gitignore ├── car_prec.stan ├── car_sparse.stan └── iar_sparse.stan /.gitignore: -------------------------------------------------------------------------------- 1 | .Rproj.user 2 | .Rhistory 3 | .RData 4 | .Ruserdata 5 | *.rda 6 | README.html 7 | blogpost.html 8 | *.rds 9 | 10 | -------------------------------------------------------------------------------- /CARstan.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | 15 | AutoAppendNewline: Yes 16 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM r-base 2 | MAINTAINER Max Joseph maxwellbjoseph@gmail.com 3 | 4 | RUN apt-get update \ 5 | && apt-get install -y --no-install-recommends \ 6 | build-essential libssl-dev pandoc 7 | 8 | RUN install2.r --error \ 9 | ggmcmc \ 10 | gpclib \ 11 | knitr \ 12 | maptools \ 13 | rmarkdown \ 14 | rstan \ 15 | spdep 16 | 17 | COPY . /home/docker/CARstan 18 | WORKDIR /home/docker/CARstan 19 | 20 | CMD ["Rscript", "-e", "rmarkdown::render('README.Rmd')"] 21 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2016, Maxwell B. Joseph 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 5 | 6 | 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 7 | 8 | 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 9 | 10 | 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. 11 | 12 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 13 | -------------------------------------------------------------------------------- /README.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Exact sparse CAR models in Stan" 3 | author: "Max Joseph" 4 | date: "August 20, 2016" 5 | output: 6 | html_document: 7 | keep_md: true 8 | --- 9 | 10 | 11 | This document details sparse exact conditional autoregressive (CAR) models in Stan as an extension of previous work on approximate sparse CAR models in Stan. 12 | Sparse representations seem to give order of magnitude efficiency gains, scaling better for large spatial data sets. 13 | 14 | ## CAR priors for spatial random effects 15 | 16 | Conditional autoregressive (CAR) models are popular as prior distributions for spatial random effects with areal spatial data. 17 | If we have a random quantity $\phi = (\phi_1, \phi_2, ..., \phi_n)'$ at $n$ areal locations, the CAR model is often expressed via full conditional distributions: 18 | 19 | $$\phi_i \mid \phi_j, j \neq i \sim N(\alpha \sum_{j = 1}^n b_{ij} \phi_j, \tau_i^{-1})$$ 20 | 21 | where $\tau_i$ is a spatially varying precision parameter, and $b_{ii} = 0$. 22 | 23 | By Brook's Lemma, the joint distribution of $\phi$ is then: 24 | 25 | $$\phi \sim N(0, [D_\tau (I - \alpha B)]^{-1}).$$ 26 | 27 | If we assume the following: 28 | 29 | - $D_\tau = \tau D$ 30 | - $D = diag(m_i)$: an $n \times n$ diagonal matrix with $m_i$ = the number of neighbors for location $i$ 31 | - $I$: an $n \times n$ identity matrix 32 | - $\alpha$: a parameter that controls spatial dependence ($\alpha = 0$ implies spatial independence, and $\alpha = 1$ collapses to an *intrisnic conditional autoregressive* (IAR) specification) 33 | - $B = D^{-1} W$: the scaled adjacency matrix 34 | - $W$: the adjacency matrix ($w_{ii} = 0, w_{ij} = 1$ if $i$ is a neighbor of $j$, and $w_{ij}=0$ otherwise) 35 | 36 | then the CAR prior specification simplifies to: 37 | 38 | $$\phi \sim N(0, [\tau (D - \alpha W)]^{-1}).$$ 39 | 40 | The $\alpha$ parameter ensures propriety of the joint distrbution of $\phi$ as long as $| \alpha | < 1$ (Gelfand & Vounatsou 2003). 41 | However, $\alpha$ is often taken as 1, leading to the IAR specification which creates a singular precision matrix and an improper prior distribution. 42 | 43 | ## A Poisson specification 44 | 45 | Suppose we have aggregated count data $y_1, y_2, ..., y_n$ at $n$ locations, and we expect that neighboring locations will have similar counts. 46 | With a Poisson likelihood: 47 | 48 | $$y_i \sim \text{Poisson}(\text{exp}(X_{i} \beta + \phi_i + \log(\text{offset}_i)))$$ 49 | 50 | where $X_i$ is a design vector (the $i^{th}$ row from a design matrix), $\beta$ is a vector of coefficients, $\phi_i$ is a spatial adjustment, and $\log(\text{offset}_i)$ accounts for differences in expected values or exposures at the spatial units (popular choices include area for physical processes, or population size for disease applications). 51 | 52 | If we specify a proper CAR prior for $\phi$, then we have that $\phi \sim \text{N}(0, [\tau (D - \alpha W)]^{-1})$ where $\tau (D - \alpha W)$ is the precision matrix $\Sigma^{-1}$. 53 | A complete Bayesian specification would include priors for the remaining parameters $\alpha$, $\tau$, and $\beta$, such that our posterior distribution is: 54 | 55 | $$p(\phi, \beta, \alpha, \tau \mid y) \propto p(y \mid \beta, \phi) p(\phi \mid \alpha, \tau) p(\alpha) p(\tau) p(\beta)$$ 56 | 57 | ## Example: Scottish lip cancer data 58 | 59 | To demonstrate this approach we'll use the Scottish lip cancer data example (some documentation [here](https://cran.r-project.org/web/packages/CARBayesdata/CARBayesdata.pdf)). 60 | This data set includes observed lip cancer case counts at 56 spatial units in Scotland, with an expected number of cases to be used as an offset, and an area-specific continuous covariate that represents the proportion of the population employed in agriculture, fishing, or forestry. 61 | The model structure is identical to the Poisson model outlined above. 62 | 63 | ```{r make-scotland-map, echo = FALSE, message = FALSE} 64 | library(dplyr) 65 | library(ggplot2) 66 | library(sf) 67 | 68 | scotlips <- st_read("/vsicurl/https://github.com/mbjoseph/CARstan/raw/master/data/scotland.shp", crs=27700) 69 | 70 | scotlips |> 71 | ggplot() + 72 | geom_sf(aes(fill = Observed)) + 73 | scale_fill_gradientn('Lip cancer cases', colors = topo.colors(3)) 74 | ``` 75 | 76 | Let's start by loading packages and data, specifying the number of MCMC iterations and chains. 77 | 78 | ```{r load-lip-cancer-data, message=FALSE} 79 | library(ggmcmc) 80 | library(bayesplot) 81 | library(cmdstanr) 82 | library(posterior) 83 | source('https://raw.githubusercontent.com/mbjoseph/CARstan/master/data/scotland_lip_cancer.RData') 84 | ``` 85 | 86 | To fit the full model, we'll pull objects loaded with our Scotland lip cancer data. 87 | I'll use `model.matrix` to generate a design matrix, centering and scaling the continuous covariate `x` to reduce correlation between the intercept and slope estimates. 88 | 89 | ```{r make-adjacency-matrix} 90 | W <- A # adjacency matrix 91 | scaled_x <- c(scale(x)) 92 | X <- model.matrix(~scaled_x) 93 | 94 | full_d <- list(n = nrow(X), # number of observations 95 | p = ncol(X), # number of coefficients 96 | X = X, # design matrix 97 | y = O, # observed number of cases 98 | log_offset = log(E), # log(expected) num. cases 99 | W = W) # adjacency matrix 100 | ``` 101 | 102 | #### Stan implementation: CAR with `multi_normal_prec` 103 | 104 | Our model statement mirrors the structure outlined above, with explicit normal and gamma priors on $\beta$ and $\tau$ respectively, and a $\text{Uniform}(0, 1)$ prior for $\alpha$. 105 | The prior on $\phi$ is specified via the `multi_normal_prec` function, passing in $\tau (D - \alpha W)$ as the precision matrix. 106 | 107 | ```{r print-stan-prec-model, comment='', echo = FALSE} 108 | cat(readLines('stan/car_prec.stan'), sep = '\n') 109 | ``` 110 | 111 | Fitting the model with `rstan`: 112 | 113 | ```{r fit-prec-model} 114 | # Define MCMC parameters 115 | niter <- 1E4 # definitely overkill, but good for comparison 116 | nchains <- 4 117 | 118 | mod <- cmdstan_model('stan/car_prec.stan') 119 | 120 | full_fit <- mod$sample( 121 | data = full_d, 122 | parallel_chains = nchains, 123 | iter_sampling = niter, 124 | show_messages = FALSE, 125 | show_exceptions = FALSE 126 | ) 127 | 128 | print(full_fit$summary(c('beta', 'tau', 'alpha', 'lp__'))) 129 | 130 | # visualize results 131 | to_plot <- c('beta', 'tau', 'alpha', 'phi[1]', 'phi[2]', 'phi[3]', 'lp__') 132 | mcmc_trace(full_fit$draws(to_plot)) 133 | ``` 134 | 135 | ### A more efficient sparse representation 136 | 137 | Although we could specify our multivariate normal prior for $\phi$ directly in Stan via `multi_normal_prec`, as we did above, in this case we will accrue computational efficiency gains by manually specifying $p(\phi \mid \tau, \alpha)$ directly via the log probability accumulator. 138 | The log probability of $\phi$ is: 139 | 140 | $$\log(p(\phi \mid \tau, \alpha)) = - \frac{n}{2} \log(2 \pi) + \frac{1}{2} \log(\text{det}( \Sigma^{-1})) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 141 | 142 | In Stan, we only need the log posterior up to an additive constant so we can drop the first term. 143 | Then, substituting $\tau (D - \alpha W)$ for $\Sigma^{-1}$: 144 | 145 | $$\frac{1}{2} \log(\text{det}(\tau (D - \alpha W))) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 146 | 147 | $$ = \frac{1}{2} \log(\tau ^ n \text{det}(D - \alpha W)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 148 | 149 | $$ = \frac{n}{2} \log(\tau) + \frac{1}{2} \log(\text{det}(D - \alpha W)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 150 | 151 | There are two ways that we can accrue computational efficiency gains: 152 | 153 | 1. Sparse representations of $\Sigma^{-1}$ to expedite computation of $\phi^T \Sigma^{-1} \phi$ (this work was done by Kyle foreman previously, e.g., https://groups.google.com/d/topic/stan-users/M7T7EIlyhoo/discussion). 154 | 155 | 2. Efficient computation of the determinant. Jin, Carlin, and Banerjee (2005) show that: 156 | 157 | $$\text{det}(D - \alpha W) \propto \prod_{i = 1}^n (1 - \alpha \lambda_i)$$ 158 | 159 | where $\lambda_1, ..., \lambda_n$ are the eigenvalues of $D^{-\frac{1}{2}} W D^{-\frac{1}{2}}$, which can be computed ahead of time and passed in as data. 160 | Because we only need the log posterior up to an additive constant, we can use this result which is proportional up to some multiplicative constant $c$: 161 | 162 | $$\frac{n}{2} \log(\tau) + \frac{1}{2} \log(c \prod_{i = 1}^n (1 - \alpha \lambda_i)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 163 | 164 | $$= \frac{n}{2} \log(\tau) + \frac{1}{2} \log(c) + \frac{1}{2} \log(\prod_{i = 1}^n (1 - \alpha \lambda_i)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 165 | 166 | Again dropping additive constants: 167 | 168 | $$\frac{n}{2} \log(\tau) + \frac{1}{2} \log(\prod_{i = 1}^n (1 - \alpha \lambda_i)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 169 | 170 | $$= \frac{n}{2} \log(\tau) + \frac{1}{2} \sum_{i = 1}^n \log(1 - \alpha \lambda_i) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 171 | 172 | ### Stan implementation: sparse CAR 173 | 174 | In the Stan model statement's `transformed data` block, we compute $\lambda_1, ..., \lambda_n$ (the eigenvalues of $D^{-\frac{1}{2}} W D^{-\frac{1}{2}}$), and generate a sparse representation for W (`Wsparse`), which is assumed to be symmetric, such that the adjacency relationships can be represented in a two column matrix where each row is an adjacency relationship between two sites. 175 | 176 | The Stan model statement for the sparse implementation never constructs the precision matrix, and does not call any of the `multi_normal*` functions. 177 | Instead, we use define a `sparse_car_lpdf()` function and use it in the model block. 178 | 179 | ```{r print-sparse-model, comment='', echo = FALSE} 180 | cat(readLines('stan/car_sparse.stan'), sep = '\n') 181 | ``` 182 | 183 | Fitting the model: 184 | 185 | ```{r fit-sparse-model} 186 | sp_d <- list(n = nrow(X), # number of observations 187 | p = ncol(X), # number of coefficients 188 | X = X, # design matrix 189 | y = O, # observed number of cases 190 | log_offset = log(E), # log(expected) num. cases 191 | W_n = sum(W) / 2, # number of neighbor pairs 192 | W = W) # adjacency matrix 193 | 194 | sp_fit <- cmdstan_model('stan/car_sparse.stan')$sample( 195 | data = sp_d, 196 | parallel_chains = nchains, 197 | iter_sampling = niter, 198 | show_messages = FALSE, 199 | show_exceptions = FALSE 200 | ) 201 | 202 | 203 | print(sp_fit$summary(c('beta', 'tau', 'alpha', 'lp__'))) 204 | 205 | mcmc_trace(sp_fit$draws(to_plot)) 206 | ``` 207 | 208 | ### MCMC Efficiency comparison 209 | 210 | The main quantity of interest is the effective number of samples per unit time. 211 | Sparsity gives us an order of magnitude or so gains, mostly via reductions in run time. 212 | 213 | ```{r make-mcmc-efficiency-table, echo = FALSE} 214 | library(knitr) 215 | efficiency <- data.frame( 216 | model = c('full', 'sparse'), 217 | n_eff = c( 218 | ess_bulk(full_fit$draws("lp__")), 219 | ess_bulk(sp_fit$draws("lp__")) 220 | ), 221 | elapsed_time = c( 222 | full_fit$time()$total, 223 | sp_fit$time()$total 224 | ) 225 | ) %>% 226 | mutate(n_eff_per_sec = n_eff / elapsed_time) 227 | 228 | names(efficiency) <- c('Model', 'Number of effective samples', 'Elapsed time (sec)', 229 | 'Effective samples / sec)') 230 | kable(efficiency) 231 | ``` 232 | 233 | ### Posterior distribution comparison 234 | 235 | Let's compare the estimates to make sure that we get the same answer with both approaches. 236 | In this case, I've used more MCMC iterations than we would typically need in to get a better estimate of the tails of each marginal posterior distribution so that we can compare the 95% credible intervals among the two approaches. 237 | 238 | ```{r compare-parameter-estimates, echo = FALSE, message = FALSE, warning=FALSE} 239 | post_full <- full_fit$draws("phi", format = "df") 240 | post_full$model <- 'full' 241 | post_sp <- sp_fit$draws("phi", format = "df") 242 | post_sp$model <- 'sparse' 243 | post <- full_join(post_full, post_sp) 244 | 245 | psumm <- post %>% 246 | tidyr::pivot_longer(starts_with("phi")) |> 247 | group_by(model, name) %>% 248 | summarize( 249 | median = median(value), 250 | lo = quantile(value, .025), 251 | hi = quantile(value, .975), 252 | .groups = "drop" 253 | ) |> 254 | tidyr::pivot_wider(names_from = "model", values_from = c("median", "lo", "hi")) 255 | 256 | # compare estimated spatial random effects 257 | psumm %>% 258 | ggplot(aes(x = median_full, y = median_sparse)) + 259 | geom_point() + 260 | geom_segment(aes(x = lo_full, xend = hi_full, yend = median_sparse)) + 261 | geom_segment(aes(xend = median_full, y = lo_sparse, yend = hi_sparse)) + 262 | geom_abline(linetype = "dashed") + 263 | xlab('Full CAR spatial effect') + 264 | ylab("Sparse CAR spatial effect") + 265 | ggtitle('Comparison on random effect estimates') 266 | ``` 267 | 268 | 269 | The two approaches give the same answers (more or less, with small differences arising due to MCMC sampling error). 270 | 271 | ## Postscript: sparse IAR specification 272 | 273 | 2023 update: for a much more comprehensive treatment of IAR models, see Morris, Mitzi, Katherine Wheeler-Martin, Dan Simpson, Stephen J. Mooney, Andrew Gelman, and Charles DiMaggio. "Bayesian hierarchical spatial models: Implementing the Besag York Mollié model in stan." Spatial and spatio-temporal epidemiology 31 (2019): 100301. https://doi.org/10.1016/j.sste.2019.100301 274 | 275 | Although the IAR prior for $\phi$ that results from $\alpha = 1$ is improper, it remains popular (Besag, York, and Mollie, 1991). 276 | In practice, these models are typically fit with a sum to zero constraints: $\sum_{i\text{ in connected coponent}} \phi_i = 0$ for each connected component of the graph. This allows us to interpret both the overall mean and the component-wise means. 277 | 278 | With $\alpha$ fixed to one, we have: 279 | 280 | $$\log(p(\phi \mid \tau)) = - \frac{n}{2} \log(2 \pi) + \frac{1}{2} \log(\text{det}^*(\tau (D - W))) - \frac{1}{2} \phi^T \tau (D - W) \phi$$ 281 | 282 | $$ = - \frac{n}{2} \log(2 \pi) + \frac{1}{2} \log(\tau^{n-k} \text{det}^*(D - W)) - \frac{1}{2} \phi^T \tau (D - W) \phi$$ 283 | 284 | $$ = - \frac{n}{2} \log(2 \pi) + \frac{1}{2} \log(\tau^{n-k}) + \frac{1}{2} \log(\text{det}^*(D - W)) - \frac{1}{2} \phi^T \tau (D - W) \phi$$ 285 | 286 | Here $\text{det}^*(A)$ is the generalized determinant of the square matrix $A$ defined as the product of its non-zero eigenvalues, and $k$ is the number of connected components in the graph. For the Scottish Lip Cancer data, there is only one connected component and $k=1$. The reason that we need to use the generalized determinant is that the precision matrix is, by definition, singular in intrinsic models as the support of the Gaussian distribution is on a subspace with fewer than $n$ dimensions. For the classical ICAR(1) model, we know that the directions correpsonding to the zero eigenvalues are exactly the vectors that are constant on each connected component of the graph and hence $k$ is the number of connected components. 287 | 288 | 289 | Dropping additive constants, the quantity to increment becomes: 290 | 291 | $$ \frac{1}{2} \log(\tau^{n-k}) - \frac{1}{2} \phi^T \tau (D - W) \phi$$ 292 | 293 | And the corresponding Stan syntax would be: 294 | 295 | ```{r print-iar-model, comment='', echo = FALSE} 296 | cat(readLines('stan/iar_sparse.stan'), sep = '\n') 297 | ``` 298 | 299 | ## References 300 | 301 | Besag, Julian, Jeremy York, and Annie Mollié. "Bayesian image restoration, with two applications in spatial statistics." Annals of the institute of statistical mathematics 43.1 (1991): 1-20. 302 | 303 | Gelfand, Alan E., and Penelope Vounatsou. "Proper multivariate conditional autoregressive models for spatial data analysis." Biostatistics 4.1 (2003): 11-15. 304 | 305 | Jin, Xiaoping, Bradley P. Carlin, and Sudipto Banerjee. "Generalized hierarchical multivariate CAR models for areal data." Biometrics 61.4 (2005): 950-961. 306 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: "Exact sparse CAR models in Stan" 3 | author: "Max Joseph" 4 | date: "August 20, 2016" 5 | output: 6 | html_document: 7 | keep_md: true 8 | --- 9 | 10 | 11 | This document details sparse exact conditional autoregressive (CAR) models in Stan as an extension of previous work on approximate sparse CAR models in Stan. 12 | Sparse representations seem to give order of magnitude efficiency gains, scaling better for large spatial data sets. 13 | 14 | ## CAR priors for spatial random effects 15 | 16 | Conditional autoregressive (CAR) models are popular as prior distributions for spatial random effects with areal spatial data. 17 | If we have a random quantity $\phi = (\phi_1, \phi_2, ..., \phi_n)'$ at $n$ areal locations, the CAR model is often expressed via full conditional distributions: 18 | 19 | $$\phi_i \mid \phi_j, j \neq i \sim N(\alpha \sum_{j = 1}^n b_{ij} \phi_j, \tau_i^{-1})$$ 20 | 21 | where $\tau_i$ is a spatially varying precision parameter, and $b_{ii} = 0$. 22 | 23 | By Brook's Lemma, the joint distribution of $\phi$ is then: 24 | 25 | $$\phi \sim N(0, [D_\tau (I - \alpha B)]^{-1}).$$ 26 | 27 | If we assume the following: 28 | 29 | - $D_\tau = \tau D$ 30 | - $D = diag(m_i)$: an $n \times n$ diagonal matrix with $m_i$ = the number of neighbors for location $i$ 31 | - $I$: an $n \times n$ identity matrix 32 | - $\alpha$: a parameter that controls spatial dependence ($\alpha = 0$ implies spatial independence, and $\alpha = 1$ collapses to an *intrisnic conditional autoregressive* (IAR) specification) 33 | - $B = D^{-1} W$: the scaled adjacency matrix 34 | - $W$: the adjacency matrix ($w_{ii} = 0, w_{ij} = 1$ if $i$ is a neighbor of $j$, and $w_{ij}=0$ otherwise) 35 | 36 | then the CAR prior specification simplifies to: 37 | 38 | $$\phi \sim N(0, [\tau (D - \alpha W)]^{-1}).$$ 39 | 40 | The $\alpha$ parameter ensures propriety of the joint distrbution of $\phi$ as long as $| \alpha | < 1$ (Gelfand & Vounatsou 2003). 41 | However, $\alpha$ is often taken as 1, leading to the IAR specification which creates a singular precision matrix and an improper prior distribution. 42 | 43 | ## A Poisson specification 44 | 45 | Suppose we have aggregated count data $y_1, y_2, ..., y_n$ at $n$ locations, and we expect that neighboring locations will have similar counts. 46 | With a Poisson likelihood: 47 | 48 | $$y_i \sim \text{Poisson}(\text{exp}(X_{i} \beta + \phi_i + \log(\text{offset}_i)))$$ 49 | 50 | where $X_i$ is a design vector (the $i^{th}$ row from a design matrix), $\beta$ is a vector of coefficients, $\phi_i$ is a spatial adjustment, and $\log(\text{offset}_i)$ accounts for differences in expected values or exposures at the spatial units (popular choices include area for physical processes, or population size for disease applications). 51 | 52 | If we specify a proper CAR prior for $\phi$, then we have that $\phi \sim \text{N}(0, [\tau (D - \alpha W)]^{-1})$ where $\tau (D - \alpha W)$ is the precision matrix $\Sigma^{-1}$. 53 | A complete Bayesian specification would include priors for the remaining parameters $\alpha$, $\tau$, and $\beta$, such that our posterior distribution is: 54 | 55 | $$p(\phi, \beta, \alpha, \tau \mid y) \propto p(y \mid \beta, \phi) p(\phi \mid \alpha, \tau) p(\alpha) p(\tau) p(\beta)$$ 56 | 57 | ## Example: Scottish lip cancer data 58 | 59 | To demonstrate this approach we'll use the Scottish lip cancer data example (some documentation [here](https://cran.r-project.org/web/packages/CARBayesdata/CARBayesdata.pdf)). 60 | This data set includes observed lip cancer case counts at 56 spatial units in Scotland, with an expected number of cases to be used as an offset, and an area-specific continuous covariate that represents the proportion of the population employed in agriculture, fishing, or forestry. 61 | The model structure is identical to the Poisson model outlined above. 62 | 63 | 64 | ``` 65 | ## Reading layer `scotland' from data source 66 | ## `/vsicurl/https://github.com/mbjoseph/CARstan/raw/master/data/scotland.shp' 67 | ## using driver `ESRI Shapefile' 68 | ## Simple feature collection with 56 features and 9 fields 69 | ## Geometry type: MULTIPOLYGON 70 | ## Dimension: XY 71 | ## Bounding box: xmin: 7150.759 ymin: 529557.2 xmax: 468393.4 ymax: 1218479 72 | ## Projected CRS: OSGB36 / British National Grid 73 | ``` 74 | 75 | ![](README_files/figure-html/make-scotland-map-1.png) 76 | 77 | Let's start by loading packages and data, specifying the number of MCMC iterations and chains. 78 | 79 | 80 | ```r 81 | library(ggmcmc) 82 | library(bayesplot) 83 | library(cmdstanr) 84 | library(posterior) 85 | source('https://raw.githubusercontent.com/mbjoseph/CARstan/master/data/scotland_lip_cancer.RData') 86 | ``` 87 | 88 | To fit the full model, we'll pull objects loaded with our Scotland lip cancer data. 89 | I'll use `model.matrix` to generate a design matrix, centering and scaling the continuous covariate `x` to reduce correlation between the intercept and slope estimates. 90 | 91 | 92 | ```r 93 | W <- A # adjacency matrix 94 | scaled_x <- c(scale(x)) 95 | X <- model.matrix(~scaled_x) 96 | 97 | full_d <- list(n = nrow(X), # number of observations 98 | p = ncol(X), # number of coefficients 99 | X = X, # design matrix 100 | y = O, # observed number of cases 101 | log_offset = log(E), # log(expected) num. cases 102 | W = W) # adjacency matrix 103 | ``` 104 | 105 | #### Stan implementation: CAR with `multi_normal_prec` 106 | 107 | Our model statement mirrors the structure outlined above, with explicit normal and gamma priors on $\beta$ and $\tau$ respectively, and a $\text{Uniform}(0, 1)$ prior for $\alpha$. 108 | The prior on $\phi$ is specified via the `multi_normal_prec` function, passing in $\tau (D - \alpha W)$ as the precision matrix. 109 | 110 | 111 | ``` 112 | data { 113 | int n; 114 | int p; 115 | matrix[n, p] X; 116 | array[n] int y; 117 | vector[n] log_offset; 118 | matrix[n, n] W; 119 | } 120 | transformed data { 121 | vector[n] zeros; 122 | matrix[n, n] D; 123 | { 124 | vector[n] W_rowsums; 125 | for (i in 1 : n) { 126 | W_rowsums[i] = sum(W[i, : ]); 127 | } 128 | D = diag_matrix(W_rowsums); 129 | } 130 | zeros = rep_vector(0, n); 131 | } 132 | parameters { 133 | vector[p] beta; 134 | vector[n] phi; 135 | real tau; 136 | real alpha; 137 | } 138 | model { 139 | phi ~ multi_normal_prec(zeros, tau * (D - alpha * W)); 140 | beta ~ normal(0, 1); 141 | tau ~ gamma(2, 2); 142 | y ~ poisson_log(X * beta + phi + log_offset); 143 | } 144 | ``` 145 | 146 | Fitting the model with `rstan`: 147 | 148 | 149 | ```r 150 | # Define MCMC parameters 151 | niter <- 1E4 # definitely overkill, but good for comparison 152 | nchains <- 4 153 | 154 | mod <- cmdstan_model('stan/car_prec.stan') 155 | 156 | full_fit <- mod$sample( 157 | data = full_d, 158 | parallel_chains = nchains, 159 | iter_sampling = niter, 160 | show_messages = FALSE, 161 | show_exceptions = FALSE 162 | ) 163 | 164 | print(full_fit$summary(c('beta', 'tau', 'alpha', 'lp__'))) 165 | ``` 166 | 167 | ``` 168 | ## # A tibble: 5 × 10 169 | ## variable mean median sd mad q5 q95 rhat ess_bulk 170 | ## 171 | ## 1 beta[1] -0.0156 -0.0103 0.285 0.232 -0.467 0.422 1.01 1026. 172 | ## 2 beta[2] 0.270 0.271 0.0942 0.0931 0.114 0.423 1.00 7961. 173 | ## 3 tau 1.66 1.59 0.508 0.476 0.950 2.58 1.00 9844. 174 | ## 4 alpha 0.934 0.952 0.0617 0.0409 0.818 0.993 1.00 3813. 175 | ## 5 lp__ 821. 821. 6.70 6.68 809. 831. 1.00 8610. 176 | ## # ℹ 1 more variable: ess_tail 177 | ``` 178 | 179 | ```r 180 | # visualize results 181 | to_plot <- c('beta', 'tau', 'alpha', 'phi[1]', 'phi[2]', 'phi[3]', 'lp__') 182 | mcmc_trace(full_fit$draws(to_plot)) 183 | ``` 184 | 185 | ![](README_files/figure-html/fit-prec-model-1.png) 186 | 187 | ### A more efficient sparse representation 188 | 189 | Although we could specify our multivariate normal prior for $\phi$ directly in Stan via `multi_normal_prec`, as we did above, in this case we will accrue computational efficiency gains by manually specifying $p(\phi \mid \tau, \alpha)$ directly via the log probability accumulator. 190 | The log probability of $\phi$ is: 191 | 192 | $$\log(p(\phi \mid \tau, \alpha)) = - \frac{n}{2} \log(2 \pi) + \frac{1}{2} \log(\text{det}( \Sigma^{-1})) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 193 | 194 | In Stan, we only need the log posterior up to an additive constant so we can drop the first term. 195 | Then, substituting $\tau (D - \alpha W)$ for $\Sigma^{-1}$: 196 | 197 | $$\frac{1}{2} \log(\text{det}(\tau (D - \alpha W))) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 198 | 199 | $$ = \frac{1}{2} \log(\tau ^ n \text{det}(D - \alpha W)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 200 | 201 | $$ = \frac{n}{2} \log(\tau) + \frac{1}{2} \log(\text{det}(D - \alpha W)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 202 | 203 | There are two ways that we can accrue computational efficiency gains: 204 | 205 | 1. Sparse representations of $\Sigma^{-1}$ to expedite computation of $\phi^T \Sigma^{-1} \phi$ (this work was done by Kyle foreman previously, e.g., https://groups.google.com/d/topic/stan-users/M7T7EIlyhoo/discussion). 206 | 207 | 2. Efficient computation of the determinant. Jin, Carlin, and Banerjee (2005) show that: 208 | 209 | $$\text{det}(D - \alpha W) \propto \prod_{i = 1}^n (1 - \alpha \lambda_i)$$ 210 | 211 | where $\lambda_1, ..., \lambda_n$ are the eigenvalues of $D^{-\frac{1}{2}} W D^{-\frac{1}{2}}$, which can be computed ahead of time and passed in as data. 212 | Because we only need the log posterior up to an additive constant, we can use this result which is proportional up to some multiplicative constant $c$: 213 | 214 | $$\frac{n}{2} \log(\tau) + \frac{1}{2} \log(c \prod_{i = 1}^n (1 - \alpha \lambda_i)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 215 | 216 | $$= \frac{n}{2} \log(\tau) + \frac{1}{2} \log(c) + \frac{1}{2} \log(\prod_{i = 1}^n (1 - \alpha \lambda_i)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 217 | 218 | Again dropping additive constants: 219 | 220 | $$\frac{n}{2} \log(\tau) + \frac{1}{2} \log(\prod_{i = 1}^n (1 - \alpha \lambda_i)) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 221 | 222 | $$= \frac{n}{2} \log(\tau) + \frac{1}{2} \sum_{i = 1}^n \log(1 - \alpha \lambda_i) - \frac{1}{2} \phi^T \Sigma^{-1} \phi$$ 223 | 224 | ### Stan implementation: sparse CAR 225 | 226 | In the Stan model statement's `transformed data` block, we compute $\lambda_1, ..., \lambda_n$ (the eigenvalues of $D^{-\frac{1}{2}} W D^{-\frac{1}{2}}$), and generate a sparse representation for W (`Wsparse`), which is assumed to be symmetric, such that the adjacency relationships can be represented in a two column matrix where each row is an adjacency relationship between two sites. 227 | 228 | The Stan model statement for the sparse implementation never constructs the precision matrix, and does not call any of the `multi_normal*` functions. 229 | Instead, we use define a `sparse_car_lpdf()` function and use it in the model block. 230 | 231 | 232 | ``` 233 | functions { 234 | /** 235 | * Return the log probability of a proper conditional autoregressive (CAR) prior 236 | * with a sparse representation for the adjacency matrix 237 | * 238 | * @param phi Vector containing the parameters with a CAR prior 239 | * @param tau Precision parameter for the CAR prior (real) 240 | * @param alpha Dependence (usually spatial) parameter for the CAR prior (real) 241 | * @param W_sparse Sparse representation of adjacency matrix (int array) 242 | * @param n Length of phi (int) 243 | * @param W_n Number of adjacent pairs (int) 244 | * @param D_sparse Number of neighbors for each location (vector) 245 | * @param lambda Eigenvalues of D^{-1/2}*W*D^{-1/2} (vector) 246 | * 247 | * @return Log probability density of CAR prior up to additive constant 248 | */ 249 | real sparse_car_lpdf(vector phi, real tau, real alpha, 250 | array[,] int W_sparse, vector D_sparse, vector lambda, 251 | int n, int W_n) { 252 | row_vector[n] phit_D; // phi' * D 253 | row_vector[n] phit_W; // phi' * W 254 | vector[n] ldet_terms; 255 | 256 | phit_D = (phi .* D_sparse)'; 257 | phit_W = rep_row_vector(0, n); 258 | for (i in 1 : W_n) { 259 | phit_W[W_sparse[i, 1]] = phit_W[W_sparse[i, 1]] + phi[W_sparse[i, 2]]; 260 | phit_W[W_sparse[i, 2]] = phit_W[W_sparse[i, 2]] + phi[W_sparse[i, 1]]; 261 | } 262 | 263 | for (i in 1 : n) { 264 | ldet_terms[i] = log1m(alpha * lambda[i]); 265 | } 266 | return 0.5 267 | * (n * log(tau) + sum(ldet_terms) 268 | - tau * (phit_D * phi - alpha * (phit_W * phi))); 269 | } 270 | } 271 | data { 272 | int n; 273 | int p; 274 | matrix[n, p] X; 275 | array[n] int y; 276 | vector[n] log_offset; 277 | matrix[n, n] W; // adjacency matrix 278 | int W_n; // number of adjacent region pairs 279 | } 280 | transformed data { 281 | array[W_n, 2] int W_sparse; // adjacency pairs 282 | vector[n] D_sparse; // diagonal of D (number of neigbors for each site) 283 | vector[n] lambda; // eigenvalues of invsqrtD * W * invsqrtD 284 | 285 | { 286 | // generate sparse representation for W 287 | int counter; 288 | counter = 1; 289 | // loop over upper triangular part of W to identify neighbor pairs 290 | for (i in 1 : (n - 1)) { 291 | for (j in (i + 1) : n) { 292 | if (W[i, j] == 1) { 293 | W_sparse[counter, 1] = i; 294 | W_sparse[counter, 2] = j; 295 | counter = counter + 1; 296 | } 297 | } 298 | } 299 | } 300 | for (i in 1 : n) { 301 | D_sparse[i] = sum(W[i]); 302 | } 303 | { 304 | vector[n] invsqrtD; 305 | for (i in 1 : n) { 306 | invsqrtD[i] = 1 / sqrt(D_sparse[i]); 307 | } 308 | lambda = eigenvalues_sym(quad_form(W, diag_matrix(invsqrtD))); 309 | } 310 | } 311 | parameters { 312 | vector[p] beta; 313 | vector[n] phi; 314 | real tau; 315 | real alpha; 316 | } 317 | model { 318 | phi ~ sparse_car(tau, alpha, W_sparse, D_sparse, lambda, n, W_n); 319 | beta ~ normal(0, 1); 320 | tau ~ gamma(2, 2); 321 | y ~ poisson_log(X * beta + phi + log_offset); 322 | } 323 | ``` 324 | 325 | Fitting the model: 326 | 327 | 328 | ```r 329 | sp_d <- list(n = nrow(X), # number of observations 330 | p = ncol(X), # number of coefficients 331 | X = X, # design matrix 332 | y = O, # observed number of cases 333 | log_offset = log(E), # log(expected) num. cases 334 | W_n = sum(W) / 2, # number of neighbor pairs 335 | W = W) # adjacency matrix 336 | 337 | sp_fit <- cmdstan_model('stan/car_sparse.stan')$sample( 338 | data = sp_d, 339 | parallel_chains = nchains, 340 | iter_sampling = niter, 341 | show_messages = FALSE, 342 | show_exceptions = FALSE 343 | ) 344 | 345 | 346 | print(sp_fit$summary(c('beta', 'tau', 'alpha', 'lp__'))) 347 | ``` 348 | 349 | ``` 350 | ## # A tibble: 5 × 10 351 | ## variable mean median sd mad q5 q95 rhat ess_bulk 352 | ## 353 | ## 1 beta[1] -0.0117 -0.0134 0.263 0.228 -0.436 0.422 1.00 964. 354 | ## 2 beta[2] 0.272 0.273 0.0944 0.0936 0.117 0.426 1.00 9052. 355 | ## 3 tau 1.64 1.58 0.498 0.471 0.952 2.55 1.00 11371. 356 | ## 4 alpha 0.933 0.950 0.0625 0.0418 0.814 0.992 1.00 5830. 357 | ## 5 lp__ 783. 783. 6.64 6.55 772. 793. 1.00 9875. 358 | ## # ℹ 1 more variable: ess_tail 359 | ``` 360 | 361 | ```r 362 | mcmc_trace(sp_fit$draws(to_plot)) 363 | ``` 364 | 365 | ![](README_files/figure-html/fit-sparse-model-1.png) 366 | 367 | ### MCMC Efficiency comparison 368 | 369 | The main quantity of interest is the effective number of samples per unit time. 370 | Sparsity gives us an order of magnitude or so gains, mostly via reductions in run time. 371 | 372 | 373 | |Model | Number of effective samples| Elapsed time (sec)| Effective samples / sec)| 374 | |:------|---------------------------:|------------------:|------------------------:| 375 | |full | 8533.779| 29.82268| 286.1506| 376 | |sparse | 9869.593| 2.10133| 4696.8318| 377 | 378 | ### Posterior distribution comparison 379 | 380 | Let's compare the estimates to make sure that we get the same answer with both approaches. 381 | In this case, I've used more MCMC iterations than we would typically need in to get a better estimate of the tails of each marginal posterior distribution so that we can compare the 95% credible intervals among the two approaches. 382 | 383 | ![](README_files/figure-html/compare-parameter-estimates-1.png) 384 | 385 | 386 | The two approaches give the same answers (more or less, with small differences arising due to MCMC sampling error). 387 | 388 | ## Postscript: sparse IAR specification 389 | 390 | 2023 update: for a much more comprehensive treatment of IAR models, see Morris, Mitzi, Katherine Wheeler-Martin, Dan Simpson, Stephen J. Mooney, Andrew Gelman, and Charles DiMaggio. "Bayesian hierarchical spatial models: Implementing the Besag York Mollié model in stan." Spatial and spatio-temporal epidemiology 31 (2019): 100301. https://doi.org/10.1016/j.sste.2019.100301 391 | 392 | Although the IAR prior for $\phi$ that results from $\alpha = 1$ is improper, it remains popular (Besag, York, and Mollie, 1991). 393 | In practice, these models are typically fit with a sum to zero constraints: $\sum_{i\text{ in connected coponent}} \phi_i = 0$ for each connected component of the graph. This allows us to interpret both the overall mean and the component-wise means. 394 | 395 | With $\alpha$ fixed to one, we have: 396 | 397 | $$\log(p(\phi \mid \tau)) = - \frac{n}{2} \log(2 \pi) + \frac{1}{2} \log(\text{det}^*(\tau (D - W))) - \frac{1}{2} \phi^T \tau (D - W) \phi$$ 398 | 399 | $$ = - \frac{n}{2} \log(2 \pi) + \frac{1}{2} \log(\tau^{n-k} \text{det}^*(D - W)) - \frac{1}{2} \phi^T \tau (D - W) \phi$$ 400 | 401 | $$ = - \frac{n}{2} \log(2 \pi) + \frac{1}{2} \log(\tau^{n-k}) + \frac{1}{2} \log(\text{det}^*(D - W)) - \frac{1}{2} \phi^T \tau (D - W) \phi$$ 402 | 403 | Here $\text{det}^*(A)$ is the generalized determinant of the square matrix $A$ defined as the product of its non-zero eigenvalues, and $k$ is the number of connected components in the graph. For the Scottish Lip Cancer data, there is only one connected component and $k=1$. The reason that we need to use the generalized determinant is that the precision matrix is, by definition, singular in intrinsic models as the support of the Gaussian distribution is on a subspace with fewer than $n$ dimensions. For the classical ICAR(1) model, we know that the directions correpsonding to the zero eigenvalues are exactly the vectors that are constant on each connected component of the graph and hence $k$ is the number of connected components. 404 | 405 | 406 | Dropping additive constants, the quantity to increment becomes: 407 | 408 | $$ \frac{1}{2} \log(\tau^{n-k}) - \frac{1}{2} \phi^T \tau (D - W) \phi$$ 409 | 410 | And the corresponding Stan syntax would be: 411 | 412 | 413 | ``` 414 | functions { 415 | /** 416 | * Return the log probability of a proper intrinsic autoregressive (IAR) prior 417 | * with a sparse representation for the adjacency matrix 418 | * 419 | * @param phi Vector containing the parameters with a IAR prior 420 | * @param tau Precision parameter for the IAR prior (real) 421 | * @param W_sparse Sparse representation of adjacency matrix (int array) 422 | * @param n Length of phi (int) 423 | * @param W_n Number of adjacent pairs (int) 424 | * @param D_sparse Number of neighbors for each location (vector) 425 | * @param lambda Eigenvalues of D^{-1/2}*W*D^{-1/2} (vector) 426 | * 427 | * @return Log probability density of IAR prior up to additive constant 428 | */ 429 | real sparse_iar_lpdf(vector phi, real tau, array[,] int W_sparse, 430 | vector D_sparse, vector lambda, int n, int W_n) { 431 | row_vector[n] phit_D; // phi' * D 432 | row_vector[n] phit_W; // phi' * W 433 | vector[n] ldet_terms; 434 | 435 | phit_D = (phi .* D_sparse)'; 436 | phit_W = rep_row_vector(0, n); 437 | for (i in 1 : W_n) { 438 | phit_W[W_sparse[i, 1]] = phit_W[W_sparse[i, 1]] + phi[W_sparse[i, 2]]; 439 | phit_W[W_sparse[i, 2]] = phit_W[W_sparse[i, 2]] + phi[W_sparse[i, 1]]; 440 | } 441 | 442 | return 0.5 * ((n - 1) * log(tau) - tau * (phit_D * phi - (phit_W * phi))); 443 | } 444 | } 445 | data { 446 | int n; 447 | int p; 448 | matrix[n, p] X; 449 | array[n] int y; 450 | vector[n] log_offset; 451 | matrix[n, n] W; // adjacency matrix 452 | int W_n; // number of adjacent region pairs 453 | } 454 | transformed data { 455 | array[W_n, 2] int W_sparse; // adjacency pairs 456 | vector[n] D_sparse; // diagonal of D (number of neigbors for each site) 457 | vector[n] lambda; // eigenvalues of invsqrtD * W * invsqrtD 458 | 459 | { 460 | // generate sparse representation for W 461 | int counter; 462 | counter = 1; 463 | // loop over upper triangular part of W to identify neighbor pairs 464 | for (i in 1 : (n - 1)) { 465 | for (j in (i + 1) : n) { 466 | if (W[i, j] == 1) { 467 | W_sparse[counter, 1] = i; 468 | W_sparse[counter, 2] = j; 469 | counter = counter + 1; 470 | } 471 | } 472 | } 473 | } 474 | for (i in 1 : n) { 475 | D_sparse[i] = sum(W[i]); 476 | } 477 | { 478 | vector[n] invsqrtD; 479 | for (i in 1 : n) { 480 | invsqrtD[i] = 1 / sqrt(D_sparse[i]); 481 | } 482 | lambda = eigenvalues_sym(quad_form(W, diag_matrix(invsqrtD))); 483 | } 484 | } 485 | parameters { 486 | vector[p] beta; 487 | vector[n] phi_unscaled; 488 | real tau; 489 | } 490 | transformed parameters { 491 | vector[n] phi; // brute force centering 492 | phi = phi_unscaled - mean(phi_unscaled); 493 | } 494 | model { 495 | phi_unscaled ~ sparse_iar(tau, W_sparse, D_sparse, lambda, n, W_n); 496 | beta ~ normal(0, 1); 497 | tau ~ gamma(2, 2); 498 | y ~ poisson_log(X * beta + phi + log_offset); 499 | } 500 | ``` 501 | 502 | ## References 503 | 504 | Besag, Julian, Jeremy York, and Annie Mollié. "Bayesian image restoration, with two applications in spatial statistics." Annals of the institute of statistical mathematics 43.1 (1991): 1-20. 505 | 506 | Gelfand, Alan E., and Penelope Vounatsou. "Proper multivariate conditional autoregressive models for spatial data analysis." Biostatistics 4.1 (2003): 11-15. 507 | 508 | Jin, Xiaoping, Bradley P. Carlin, and Sudipto Banerjee. "Generalized hierarchical multivariate CAR models for areal data." Biometrics 61.4 (2005): 950-961. 509 | -------------------------------------------------------------------------------- /README_files/figure-html/compare-parameter-estimates-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbjoseph/CARstan/9cc9f77dd93ec40a2c53e9367ac41112a59df367/README_files/figure-html/compare-parameter-estimates-1.png -------------------------------------------------------------------------------- /README_files/figure-html/fit-prec-model-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbjoseph/CARstan/9cc9f77dd93ec40a2c53e9367ac41112a59df367/README_files/figure-html/fit-prec-model-1.png -------------------------------------------------------------------------------- /README_files/figure-html/fit-sparse-model-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbjoseph/CARstan/9cc9f77dd93ec40a2c53e9367ac41112a59df367/README_files/figure-html/fit-sparse-model-1.png -------------------------------------------------------------------------------- /README_files/figure-html/make-scotland-map-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbjoseph/CARstan/9cc9f77dd93ec40a2c53e9367ac41112a59df367/README_files/figure-html/make-scotland-map-1.png -------------------------------------------------------------------------------- /README_files/figure-html/unnamed-chunk-1-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbjoseph/CARstan/9cc9f77dd93ec40a2c53e9367ac41112a59df367/README_files/figure-html/unnamed-chunk-1-1.png -------------------------------------------------------------------------------- /blogpost.md: -------------------------------------------------------------------------------- 1 | Two weird tricks for fast conditional autoregressive models in Stan 2 | =================================================================== 3 | 4 | Conditional autoregressive (CAR) models are popular as prior distributions for spatial random effects with areal spatial data. 5 | Historically, MCMC algorithms for CAR models have benefitted from efficient Gibbs sampling via full conditional distributions for the spatial random effects. 6 | But, these conditional specifications do not work in Stan, where the joint density needs to be specified (up to a multiplicative constant). 7 | 8 | CAR models can still be implemented in Stan by specifying a multivariate normal prior on the spatial random effects, parameterized by a mean vector and a precision matrix. 9 | This works, but is slow and hard to scale to large datasets. 10 | 11 | Order(s) of magnitude speedups can be achieved by combining 1) sparse matrix multiplications from Kyle Foreman (outlined [on the stan-users mailing list](https://groups.google.com/d/topic/stan-users/M7T7EIlyhoo/discussion)), and 2) a fancy determinant trick from Jin, Carlin, and Banerjee (2005). 12 | With the oft-used Scotland lip cancer dataset, the sparse CAR implementation with the NUTS (No-U-Turn Sampler) algorithm in Stan gives 120 effective samples/sec compared to 7 effective samples/sec for the precision matrix implementation. 13 | Details for these sparse exact methods can be found at https://github.com/mbjoseph/CARstan. 14 | 15 | (Max Joseph is part of the Earth Lab Analytics Hub, University of Colorado - Boulder.) 16 | 17 | ### References 18 | 19 | Jin, Xiaoping, Bradley P. Carlin, and Sudipto Banerjee. "Generalized hierarchical multivariate CAR models for areal data." Biometrics 61.4 (2005): 950-961. 20 | -------------------------------------------------------------------------------- /data/scotland.dbf: -------------------------------------------------------------------------------- 1 | _8AtSP_IDCNAMEC IDNDistrictNObservedNExpectedNpcaffNLatitudeNLongitudeN 12 Sutherland 12.00000000000000 12 5 1.80000000000000 16 58.06000000000000 4.640000000000000 13 Nairn 13.00000000000000 13 3 1.10000000000000 10 57.47000000000000 3.980000000000000 19 Inverness 19.00000000000000 19 9 5.50000000000000 7 57.24000000000000 4.730000000000000 02 Banff-Buchan 2.00000000000000 2 39 8.70000000000000 16 57.56000000000000 2.360000000000000 17 Bedenoch 17.00000000000000 17 2 1.10000000000000 10 57.06000000000000 4.090000000000000 16 Kincardine 16.00000000000000 16 9 4.60000000000000 16 57.00000000000000 3.000000000000000 21 Angus 21.00000000000000 21 16 10.50000000000000 7 56.75000000000000 2.980000000000000 50 Dundee 50.00000000000000 50 6 19.60000000000000 1 56.45000000000000 3.200000000000000 15 NE Fife 15.00000000000000 15 17 7.80000000000000 7 56.30000000000000 3.100000000000000 25 Kirkcaldy 25.00000000000000 25 19 15.50000000000000 1 56.20000000000000 3.300000000000000 26 Dunfermline 26.00000000000000 26 15 12.50000000000000 1 56.10000000000000 3.600000000000000 29 Perth-Kinross 29.00000000000000 29 16 14.40000000000000 10 56.60000000000000 4.090000000000000 43 Clackmannan 43.00000000000000 43 2 4.30000000000000 16 56.15000000000000 3.960000000000000 39 Dumbarton 39.00000000000000 39 6 7.20000000000000 16 56.15000000000000 4.990000000000000 40 Clydebank 40.00000000000000 40 4 5.30000000000000 0 56.05000000000000 4.910000000000000 52 Bearsden 52.00000000000000 52 1 3.60000000000000 0 56.15000000000000 4.640000000000000 42 Falkirk 42.00000000000000 42 8 15.80000000000000 16 56.03000000000000 4.000000000000000 51 Cumbernauld 51.00000000000000 51 1 3.40000000000000 1 56.00000000000000 4.270000000000000 34 Stirling 34.00000000000000 34 8 8.50000000000000 7 56.30000000000000 4.730000000000000 54 Strathkelvin 54.00000000000000 54 1 7.00000000000000 1 55.99000000000000 4.450000000000000 36 Inverclyde 36.00000000000000 36 9 10.10000000000000 0 55.94000000000000 4.950000000000000 46 Kilmarnock 46.00000000000000 46 3 8.20000000000000 7 55.65000000000000 4.750000000000000 41 Renfrew 41.00000000000000 41 10 18.80000000000000 1 55.88000000000000 4.820000000000000 53 Eastwood 53.00000000000000 53 1 5.70000000000000 1 55.79000000000000 4.700000000000000 49 Glasgow 49.00000000000000 49 28 88.70000000000000 0 55.90000000000000 4.550000000000000 38 Monklands 38.00000000000000 38 8 9.40000000000000 1 55.91000000000000 4.180000000000000 44 Motherwell 44.00000000000000 44 6 14.60000000000000 0 55.82000000000000 4.090000000000000 30 West Lothian 30.00000000000000 30 11 10.20000000000000 10 55.90000000000000 3.800000000000000 45 Edinburgh 45.00000000000000 45 19 50.70000000000000 1 55.93000000000000 3.400000000000000 48 Hamilton 48.00000000000000 48 3 9.30000000000000 1 55.79000000000000 4.270000000000000 47 East Kilbride 47.00000000000000 47 2 5.60000000000000 1 55.71000000000000 4.450000000000000 35 Kyle-Carrick 35.00000000000000 35 11 12.30000000000000 7 55.29000000000000 4.980000000000000 28 East Lothian 28.00000000000000 28 10 9.00000000000000 7 55.95000000000000 2.800000000000000 04 Berwickshire 4.00000000000000 4 9 2.50000000000000 24 55.76000000000000 2.400000000000000 20 Roxburgh 20.00000000000000 20 7 4.40000000000000 10 55.35000000000000 2.900000000000000 33 Midlothian 33.00000000000000 33 7 7.00000000000000 10 55.83000000000000 3.200000000000000 31 Cumnock-Doon 31.00000000000000 31 5 4.80000000000000 7 55.47000000000000 4.550000000000000 24 Clydesdale 24.00000000000000 24 7 5.60000000000000 7 55.63000000000000 3.960000000000000 55 Tweeddale 55.00000000000000 55 0 4.20000000000000 16 55.68000000000000 3.380000000000000 18 Ettrick 18.00000000000000 18 7 4.20000000000000 7 55.65000000000000 2.880000000000000 56 Annandale 56.00000000000000 56 0 1.80000000000000 10 55.18000000000000 3.400000000000000 14 Wigtown 14.00000000000000 14 8 3.30000000000000 24 54.94000000000000 5.000000000000000 32 Stewartry 32.00000000000000 32 3 2.90000000000000 24 55.00000000000000 4.360000000000000 27 Nithsdale 27.00000000000000 27 7 6.00000000000000 7 55.24000000000000 4.090000000000000 10 Gorden 10.00000000000000 10 20 6.60000000000000 16 57.24000000000000 2.600000000000000 22 Aberdeen 22.00000000000000 22 31 22.70000000000000 16 57.12000000000000 2.200000000000000 06 Orkney 6.00000000000000 6 8 2.40000000000000 24 59.13000000000000 3.250000000000000 08 Shetland 8.00000000000000 8 7 2.30000000000000 7 60.24000000000000 1.430000000000000 09 Lochaber 9.00000000000000 9 6 2.00000000000000 7 56.90000000000000 5.420000000000000 03 Caithness 3.00000000000000 3 11 3.00000000000000 10 58.44000000000000 3.900000000000000 05 Ross-Cromarty 5.00000000000000 5 15 4.30000000000000 10 57.71000000000000 5.090000000000000 11 Western Isles 11.00000000000000 11 13 4.40000000000000 7 58.12000000000000 6.800000000000000 01 Skye-Lochalsh 1.00000000000000 1 9 1.40000000000000 16 57.29000000000000 5.500000000000000 07 Moray 7.00000000000000 7 26 8.10000000000000 10 57.47000000000000 3.300000000000000 23 Argyll-Bute 23.00000000000000 23 11 8.80000000000000 10 56.40000000000000 5.270000000000000 37 Cunninghame 37.00000000000000 37 11 12.70000000000000 10 55.76000000000000 5.020000000000000 -------------------------------------------------------------------------------- /data/scotland.shp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbjoseph/CARstan/9cc9f77dd93ec40a2c53e9367ac41112a59df367/data/scotland.shp -------------------------------------------------------------------------------- /data/scotland.shx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mbjoseph/CARstan/9cc9f77dd93ec40a2c53e9367ac41112a59df367/data/scotland.shx -------------------------------------------------------------------------------- /data/scotland_lip_cancer.RData: -------------------------------------------------------------------------------- 1 | # num obs 2 | N <- 56 3 | 4 | # observed 5 | O <- c( 9, 39, 11, 9, 15, 8, 26, 7, 6, 20, 6 | 13, 5, 3, 8, 17, 9, 2, 7, 9, 7, 7 | 16, 31, 11, 7, 19, 15, 7, 10, 16, 11, 8 | 5, 3, 7, 8, 11, 9, 11, 8, 6, 4, 9 | 10, 8, 2, 6, 19, 3, 2, 3, 28, 6, 10 | 1, 1, 1, 1, 0, 0) 11 | 12 | # expected 13 | E <- c( 1.4, 8.7, 3.0, 2.5, 4.3, 2.4, 8.1, 2.3, 2.0, 6.6, 14 | 4.4, 1.8, 1.1, 3.3, 7.8, 4.6, 1.1, 4.2, 5.5, 4.4, 15 | 10.5,22.7, 8.8, 5.6,15.5,12.5, 6.0, 9.0,14.4,10.2, 16 | 4.8, 2.9, 7.0, 8.5,12.3,10.1,12.7, 9.4, 7.2, 5.3, 17 | 18.8,15.8, 4.3,14.6,50.7, 8.2, 5.6, 9.3,88.7,19.6, 18 | 3.4, 3.6, 5.7, 7.0, 4.2, 1.8) 19 | 20 | # covariate 21 | x <- c(16,16,10,24,10,24,10, 7, 7,16, 22 | 7,16,10,24, 7,16,10, 7, 7,10, 23 | 7,16,10, 7, 1, 1, 7, 7,10,10, 24 | 7,24,10, 7, 7, 0,10, 1,16, 0, 25 | 1,16,16, 0, 1, 7, 1, 1, 0, 1, 26 | 1, 0, 1, 1,16,10) 27 | 28 | # adjacency matrix 29 | A <- structure(c(0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 30 | 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 31 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 32 | 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 33 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 34 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 35 | 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 36 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 37 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 38 | 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 39 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 40 | 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 41 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 42 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 43 | 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 44 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 45 | 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 46 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 47 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 48 | 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 49 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 50 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 51 | 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 52 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 53 | 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 54 | 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 55 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 56 | 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 57 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 59 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 60 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 61 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 62 | 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 63 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 64 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 65 | 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 66 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 67 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 68 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 69 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 70 | 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 71 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 72 | 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 73 | 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 74 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 75 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 76 | 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 77 | 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 78 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 79 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 80 | 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 81 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 82 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 83 | 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 84 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 85 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 86 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 87 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 88 | 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 89 | 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 90 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 91 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 92 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 93 | 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 94 | 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 95 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 96 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 97 | 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 98 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 99 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 100 | 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 101 | 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 102 | 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 103 | 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 104 | 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 105 | 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 106 | 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 107 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 108 | 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 109 | 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 110 | 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 111 | 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 112 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 113 | 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 114 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 115 | 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 116 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 117 | 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 118 | 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 119 | 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 120 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 121 | 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 122 | 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 123 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 124 | 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 125 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 126 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 127 | 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 128 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 129 | 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 130 | 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 131 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 132 | 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 133 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 134 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 135 | 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 136 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 137 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 138 | 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 139 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 140 | 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 141 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 142 | 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 143 | 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 144 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 145 | 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 146 | 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 147 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 148 | 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 149 | 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 150 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 151 | 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 152 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 153 | 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 154 | 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 155 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 156 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 157 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 158 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 159 | 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 160 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 161 | 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 162 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 163 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 164 | 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 165 | 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 166 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 167 | 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 168 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 169 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 170 | 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 171 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 172 | 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 173 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 174 | 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 175 | 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 176 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 177 | 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 178 | 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0), .Dim = c(N, N)) 179 | -------------------------------------------------------------------------------- /stan/.gitignore: -------------------------------------------------------------------------------- 1 | car_prec 2 | car_sparse 3 | -------------------------------------------------------------------------------- /stan/car_prec.stan: -------------------------------------------------------------------------------- 1 | data { 2 | int n; 3 | int p; 4 | matrix[n, p] X; 5 | array[n] int y; 6 | vector[n] log_offset; 7 | matrix[n, n] W; 8 | } 9 | transformed data { 10 | vector[n] zeros; 11 | matrix[n, n] D; 12 | { 13 | vector[n] W_rowsums; 14 | for (i in 1 : n) { 15 | W_rowsums[i] = sum(W[i, : ]); 16 | } 17 | D = diag_matrix(W_rowsums); 18 | } 19 | zeros = rep_vector(0, n); 20 | } 21 | parameters { 22 | vector[p] beta; 23 | vector[n] phi; 24 | real tau; 25 | real alpha; 26 | } 27 | model { 28 | phi ~ multi_normal_prec(zeros, tau * (D - alpha * W)); 29 | beta ~ normal(0, 1); 30 | tau ~ gamma(2, 2); 31 | y ~ poisson_log(X * beta + phi + log_offset); 32 | } 33 | -------------------------------------------------------------------------------- /stan/car_sparse.stan: -------------------------------------------------------------------------------- 1 | functions { 2 | /** 3 | * Return the log probability of a proper conditional autoregressive (CAR) prior 4 | * with a sparse representation for the adjacency matrix 5 | * 6 | * @param phi Vector containing the parameters with a CAR prior 7 | * @param tau Precision parameter for the CAR prior (real) 8 | * @param alpha Dependence (usually spatial) parameter for the CAR prior (real) 9 | * @param W_sparse Sparse representation of adjacency matrix (int array) 10 | * @param n Length of phi (int) 11 | * @param W_n Number of adjacent pairs (int) 12 | * @param D_sparse Number of neighbors for each location (vector) 13 | * @param lambda Eigenvalues of D^{-1/2}*W*D^{-1/2} (vector) 14 | * 15 | * @return Log probability density of CAR prior up to additive constant 16 | */ 17 | real sparse_car_lpdf(vector phi, real tau, real alpha, 18 | array[,] int W_sparse, vector D_sparse, vector lambda, 19 | int n, int W_n) { 20 | row_vector[n] phit_D; // phi' * D 21 | row_vector[n] phit_W; // phi' * W 22 | vector[n] ldet_terms; 23 | 24 | phit_D = (phi .* D_sparse)'; 25 | phit_W = rep_row_vector(0, n); 26 | for (i in 1 : W_n) { 27 | phit_W[W_sparse[i, 1]] = phit_W[W_sparse[i, 1]] + phi[W_sparse[i, 2]]; 28 | phit_W[W_sparse[i, 2]] = phit_W[W_sparse[i, 2]] + phi[W_sparse[i, 1]]; 29 | } 30 | 31 | for (i in 1 : n) { 32 | ldet_terms[i] = log1m(alpha * lambda[i]); 33 | } 34 | return 0.5 35 | * (n * log(tau) + sum(ldet_terms) 36 | - tau * (phit_D * phi - alpha * (phit_W * phi))); 37 | } 38 | } 39 | data { 40 | int n; 41 | int p; 42 | matrix[n, p] X; 43 | array[n] int y; 44 | vector[n] log_offset; 45 | matrix[n, n] W; // adjacency matrix 46 | int W_n; // number of adjacent region pairs 47 | } 48 | transformed data { 49 | array[W_n, 2] int W_sparse; // adjacency pairs 50 | vector[n] D_sparse; // diagonal of D (number of neigbors for each site) 51 | vector[n] lambda; // eigenvalues of invsqrtD * W * invsqrtD 52 | 53 | { 54 | // generate sparse representation for W 55 | int counter; 56 | counter = 1; 57 | // loop over upper triangular part of W to identify neighbor pairs 58 | for (i in 1 : (n - 1)) { 59 | for (j in (i + 1) : n) { 60 | if (W[i, j] == 1) { 61 | W_sparse[counter, 1] = i; 62 | W_sparse[counter, 2] = j; 63 | counter = counter + 1; 64 | } 65 | } 66 | } 67 | } 68 | for (i in 1 : n) { 69 | D_sparse[i] = sum(W[i]); 70 | } 71 | { 72 | vector[n] invsqrtD; 73 | for (i in 1 : n) { 74 | invsqrtD[i] = 1 / sqrt(D_sparse[i]); 75 | } 76 | lambda = eigenvalues_sym(quad_form(W, diag_matrix(invsqrtD))); 77 | } 78 | } 79 | parameters { 80 | vector[p] beta; 81 | vector[n] phi; 82 | real tau; 83 | real alpha; 84 | } 85 | model { 86 | phi ~ sparse_car(tau, alpha, W_sparse, D_sparse, lambda, n, W_n); 87 | beta ~ normal(0, 1); 88 | tau ~ gamma(2, 2); 89 | y ~ poisson_log(X * beta + phi + log_offset); 90 | } 91 | -------------------------------------------------------------------------------- /stan/iar_sparse.stan: -------------------------------------------------------------------------------- 1 | functions { 2 | /** 3 | * Return the log probability of a proper intrinsic autoregressive (IAR) prior 4 | * with a sparse representation for the adjacency matrix 5 | * 6 | * @param phi Vector containing the parameters with a IAR prior 7 | * @param tau Precision parameter for the IAR prior (real) 8 | * @param W_sparse Sparse representation of adjacency matrix (int array) 9 | * @param n Length of phi (int) 10 | * @param W_n Number of adjacent pairs (int) 11 | * @param D_sparse Number of neighbors for each location (vector) 12 | * @param lambda Eigenvalues of D^{-1/2}*W*D^{-1/2} (vector) 13 | * 14 | * @return Log probability density of IAR prior up to additive constant 15 | */ 16 | real sparse_iar_lpdf(vector phi, real tau, array[,] int W_sparse, 17 | vector D_sparse, vector lambda, int n, int W_n) { 18 | row_vector[n] phit_D; // phi' * D 19 | row_vector[n] phit_W; // phi' * W 20 | vector[n] ldet_terms; 21 | 22 | phit_D = (phi .* D_sparse)'; 23 | phit_W = rep_row_vector(0, n); 24 | for (i in 1 : W_n) { 25 | phit_W[W_sparse[i, 1]] = phit_W[W_sparse[i, 1]] + phi[W_sparse[i, 2]]; 26 | phit_W[W_sparse[i, 2]] = phit_W[W_sparse[i, 2]] + phi[W_sparse[i, 1]]; 27 | } 28 | 29 | return 0.5 * ((n - 1) * log(tau) - tau * (phit_D * phi - (phit_W * phi))); 30 | } 31 | } 32 | data { 33 | int n; 34 | int p; 35 | matrix[n, p] X; 36 | array[n] int y; 37 | vector[n] log_offset; 38 | matrix[n, n] W; // adjacency matrix 39 | int W_n; // number of adjacent region pairs 40 | } 41 | transformed data { 42 | array[W_n, 2] int W_sparse; // adjacency pairs 43 | vector[n] D_sparse; // diagonal of D (number of neigbors for each site) 44 | vector[n] lambda; // eigenvalues of invsqrtD * W * invsqrtD 45 | 46 | { 47 | // generate sparse representation for W 48 | int counter; 49 | counter = 1; 50 | // loop over upper triangular part of W to identify neighbor pairs 51 | for (i in 1 : (n - 1)) { 52 | for (j in (i + 1) : n) { 53 | if (W[i, j] == 1) { 54 | W_sparse[counter, 1] = i; 55 | W_sparse[counter, 2] = j; 56 | counter = counter + 1; 57 | } 58 | } 59 | } 60 | } 61 | for (i in 1 : n) { 62 | D_sparse[i] = sum(W[i]); 63 | } 64 | { 65 | vector[n] invsqrtD; 66 | for (i in 1 : n) { 67 | invsqrtD[i] = 1 / sqrt(D_sparse[i]); 68 | } 69 | lambda = eigenvalues_sym(quad_form(W, diag_matrix(invsqrtD))); 70 | } 71 | } 72 | parameters { 73 | vector[p] beta; 74 | vector[n] phi_unscaled; 75 | real tau; 76 | } 77 | transformed parameters { 78 | vector[n] phi; // brute force centering 79 | phi = phi_unscaled - mean(phi_unscaled); 80 | } 81 | model { 82 | phi_unscaled ~ sparse_iar(tau, W_sparse, D_sparse, lambda, n, W_n); 83 | beta ~ normal(0, 1); 84 | tau ~ gamma(2, 2); 85 | y ~ poisson_log(X * beta + phi + log_offset); 86 | } 87 | --------------------------------------------------------------------------------