├── LICENSE ├── README.md ├── sparse_poisson.py └── sparse_tensor.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Eliezer Silva 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # sparse-poisson-factorization 2 | Non-negative probabilistic Poisson-gamma matrix factorization and tensor CP decomposition with sparse input and internal components. 3 | This is a (more) memory efficient version of Poisson Factorization in pure python that takes advantage of the fact that inference updates for Poisson Factorization needs to be performed only on the non-zero entries of the input matrix/tensor. 4 | Both models consist on a hierarchical probabilistic model with a Poisson likelihood for the matrix entries, and Gamma distributed latent factors vectors for rows and columns (in the matrix case) or each mode (in the tensor case, effectively meaning that we are performing a non-negative CP decomposition). 5 | 6 | For the matrix case the expected value of latent factor can be accessed via the attributes `Eb` and `Et` of the poisson factorization object, while for the tensor CP decomposition the expected value of latent factors can be acessed via the attribute `Eb` indexed by the mode. 7 | 8 | ## Dependencies: 9 | - numpy_indexed 10 | - scipy 11 | - numpy 12 | - scikit-learn 13 | 14 | ## Usage 15 | 16 | The input for the train method on the matrix factorization should be a numpy array representing a sparse matrix. An array with shape (N,3), where N is the number of non-zero entries and for each non-zero entry an array with `[row, column, value]` ). 17 | A dense array equivalent for each entry would be `Matrix[row,colum]=value`. 18 | 19 | Example: 20 | 21 | `[ [row_1, col_1, matrix_entry_row_1_col_1],` 22 | 23 | ` [row_2, col_2, matrix_entry_row_2_col_2],` 24 | 25 | ` [row_3, col_3, matrix_entry_row_3_col_3],` 26 | 27 | ` ... ]` 28 | 29 | 30 | The input for the train method on the tensor factorization should be a numpy array representing a sparse vector. An array of shape (N,N_modes+1), where N is the number of non-zero entries, N_mode is the number of modes in the tensor, and for each non-zero entry an array of size N_modes+1 with `[mode1, mode2, ... , mode_N,value]`. 31 | A dense tensor array equivalent for each entry would be `Tensor[mode1, mode2, ... , mode_N]=value`. 32 | 33 | Example: 34 | 35 | `[ [mode1_1, mode2_1, ... , mode_N_1, value_1],` 36 | 37 | ` [mode1_2, mode2_2, ... , mode_N_2, value_2],` 38 | 39 | ` ... ]` 40 | 41 | 42 | To use sparse poisson matrix factorization add the following import: 43 | 44 | 45 | import sparse_poisson as sp 46 | 47 | poisson = sp.PoissonMF(n_components=15,max_iter=1000,smoothness=0.1,verbose=True,tol=0.0001,a=0.1,b=0.1) 48 | poisson.fit(X)` 49 | 50 | To use sparse poisson tensor factorization add the following import: 51 | 52 | 53 | import sparse_tensor as st 54 | 55 | poisson = st.PoissonTF(n_components=15,max_iter=1000,smoothness=0.1,verbose=True,tol=0.0001,a=0.1,b=0.1) 56 | poisson.fit(X)` 57 | 58 | 59 | This implementation is modification of the code found in [PMF](https://github.com/dawenl/stochastic_PMF/blob/master/code/pmf.py) by Dawen Liang 60 | 61 | To understand the models implemented look at the Following bibliography: 62 | - [Scalable Recommendation with Poisson Factorization](https://arxiv.org/abs/1311.1704) 63 | - [Bayesian Poisson Tensor Factorization for Inferring Multilateral Relations from Sparse Dyadic Event Counts](https://arxiv.org/abs/1506.03493) 64 | -------------------------------------------------------------------------------- /sparse_poisson.py: -------------------------------------------------------------------------------- 1 | """ 2 | Poisson Matrix Factorization using sparse representation of input matrix by: 2017-11-24 Eliezer de Souza da Silva 3 | Modification of a code created by: 2014-03-25 02:06:52 by Dawen Liang 4 | """ 5 | 6 | import sys 7 | import numpy as np 8 | from scipy import special 9 | import numpy_indexed as npi 10 | import matplotlib.pyplot as plt 11 | 12 | 13 | from sklearn.base import BaseEstimator, TransformerMixin 14 | 15 | class PoissonMF(BaseEstimator, TransformerMixin): 16 | """ Poisson matrix factorization with batch inference and sparse input matrix and internal representation """ 17 | def __init__(self, n_components=100, max_iter=100, tol=0.0005, 18 | smoothness=100, random_state=None, verbose=False,allone=False, 19 | **kwargs): 20 | """ Poisson matrix factorization 21 | Arguments 22 | --------- 23 | n_components : int 24 | Number of latent components 25 | max_iter : int 26 | Maximal number of iterations to perform 27 | tol : float 28 | The threshold on the increase of the objective to stop the 29 | iteration 30 | smoothness : int 31 | Smoothness on the initialization variational parameters 32 | random_state : int or RandomState 33 | Pseudo random number generator used for sampling 34 | verbose : bool 35 | Whether to show progress during model fitting 36 | **kwargs: dict 37 | Model hyperparameters: theta_a, theta_b, beta_a, beta_b 38 | """ 39 | self.allone=allone 40 | self.n_components = n_components 41 | self.max_iter = max_iter 42 | self.tol = tol 43 | self.smoothness = smoothness 44 | self.random_state = random_state 45 | self.verbose = verbose 46 | if type(self.random_state) is int: 47 | np.random.seed(self.random_state) 48 | elif self.random_state is not None: 49 | np.random.setstate(self.random_state) 50 | self._parse_args(**kwargs) 51 | 52 | def _parse_args(self, **kwargs): 53 | self.a1 = float(kwargs.get('theta_a', 0.1)) 54 | self.a2 = float(kwargs.get('theta_b', 0.1)) 55 | self.b1 = float(kwargs.get('beta_a', 0.1)) 56 | self.b2 = float(kwargs.get('beta_b', 0.1)) 57 | 58 | def _init_components(self, n_rows,n_cols): 59 | # variational parameters for beta 60 | self.gamma_b = self.smoothness \ 61 | * np.random.gamma(self.smoothness, 1. / self.smoothness, 62 | size=(n_rows, self.n_components)) 63 | self.rho_b = self.smoothness \ 64 | * np.random.gamma(self.smoothness, 1. / self.smoothness, 65 | size=(n_rows, self.n_components)) 66 | self.Eb, self.Elogb = _compute_expectations(self.gamma_b, self.rho_b) 67 | # variational parameters for theta 68 | self.gamma_t = self.smoothness \ 69 | * np.random.gamma(self.smoothness, 1. / self.smoothness, 70 | size=(n_cols, self.n_components)) 71 | self.rho_t = self.smoothness \ 72 | * np.random.gamma(self.smoothness, 1. / self.smoothness, 73 | size=(n_cols, self.n_components)) 74 | self.Et, self.Elogt = _compute_expectations(self.gamma_t, self.rho_t) 75 | 76 | def fit(self, X): 77 | '''Fit the model to the data in X. 78 | Parameters 79 | ---------- 80 | X : array-like, shape (n_examples, 3) 81 | Training data. 82 | Returns 83 | ------- 84 | self: object 85 | Returns the instance itself. 86 | ''' 87 | X_new=np.zeros(shape=X.shape,dtype=int) 88 | if self.allone: 89 | X_new[:, -1]=1 90 | else: 91 | X_new[:, -1]=X[:,-1] ## copy the last column 92 | unique_rows= np.unique(X[:,0]) 93 | unique_cols= np.unique(X[:,1]) 94 | d_rows = dict(zip(unique_rows,range(len(unique_rows)))) 95 | d_cols = dict(zip(unique_cols, range(len(unique_cols)))) 96 | X_new[:, 0] = np.array([d_rows[x] for x in X[:, 0]],dtype=np.int64) 97 | X_new[:, 1] = np.array([d_cols[x] for x in X[:, 1]],dtype=np.int64) 98 | self.n_rows = np.max(X_new[:,0])+1 99 | self.n_cols = np.max(X_new[:,1])+1 100 | if self.verbose: 101 | print("cols=",self.n_cols) 102 | print("rows=",self.n_rows) 103 | self.row_index = X_new[:,0] 104 | self.cols_index = X_new[:,1] 105 | self.vals_vec = X_new[:,2] 106 | self._init_components(self.n_rows,self.n_cols) #beta, theta 107 | return self._update(X_new) 108 | 109 | def transform(self, X, attr=None): 110 | '''Encode the data as a linear combination of the latent components. 111 | TODO 112 | ''' 113 | return 1 114 | def _update_phi(self,X): 115 | self.phi_var = np.zeros((X.shape[0], self.n_components)) 116 | self.phi_var = np.add(self.phi_var, np.exp(self.Elogb[self.row_index, :])) 117 | self.phi_var = np.add(self.phi_var, np.exp(self.Elogt[self.cols_index, :])) 118 | self.phi_var = np.divide(self.phi_var, np.sum(self.phi_var, axis=1)[:, np.newaxis]) 119 | self.phi_var =self.vals_vec[:,np.newaxis]*self.phi_var 120 | 121 | def _update(self, X, update_beta=True): 122 | # alternating between update latent components and weights 123 | old_bd = -np.inf 124 | elbo_lst = [] 125 | for i in range(self.max_iter): 126 | self._update_phi(X) 127 | self._update_theta(X) 128 | if update_beta: 129 | self._update_phi(X) 130 | self._update_beta(X) 131 | bound = self._bound(X) 132 | elbo_lst.append(bound) 133 | if(i > 0): 134 | improvement = abs((bound - old_bd) / (old_bd)) 135 | if self.verbose: 136 | sys.stdout.write('\r\tAfter ITERATION: %d\tObjective: %.2f\t' 137 | 'Old objective: %.2f\t' 138 | 'Improvement: %.5f' % (i, bound, old_bd, 139 | improvement)) 140 | sys.stdout.flush() 141 | if improvement < self.tol: 142 | break 143 | old_bd = bound 144 | if self.verbose: 145 | sys.stdout.write('\n') 146 | return elbo_lst 147 | 148 | def _update_theta(self, X): 149 | self.gamma_t = self.a1 + npi.group_by(self.cols_index).sum(self.phi_var)[1] 150 | self.rho_t = self.a2 + np.sum(self.Eb, axis=0, keepdims=True) 151 | self.Et, self.Elogt = _compute_expectations(self.gamma_t, self.rho_t) 152 | 153 | def _update_beta(self, X): 154 | self.gamma_b = self.b1 + npi.group_by(self.row_index).sum(self.phi_var)[1] 155 | self.rho_b = self.b2 + np.sum(self.Et, axis=0, keepdims=True) 156 | self.Eb, self.Elogb = _compute_expectations(self.gamma_b, self.rho_b) 157 | 158 | def _bound(self, X): 159 | bound = np.sum(self.phi_var*(self.Elogt[self.cols_index, :]+self.Elogb[self.row_index, :])) 160 | bound -= np.sum(self.phi_var*(np.log(self.phi_var)-np.log(X[:,2]).reshape(X.shape[0],1))) 161 | bound -= np.sum(np.inner(self.Eb,self.Et)) 162 | bound += _gamma_term(self.a1, self.a2 , 163 | self.gamma_t, self.rho_t, 164 | self.Et, self.Elogt) 165 | bound += _gamma_term(self.b1, self.b2, self.gamma_b, self.rho_b, 166 | self.Eb, self.Elogb) 167 | return bound 168 | def samplePosterior(self): 169 | latent_a = np.random.gamma(self.gamma_t,1./self.rho_t) 170 | latent_b = np.random.gamma(self.gamma_b,1./self.rho_b) 171 | return np.random.poisson(np.inner(latent_a,latent_b)) 172 | 173 | def samplePrior(self): 174 | latent_a = np.random.gamma(self.a1,1./self.a2,(self.n_cols,self.n_components)) 175 | latent_b = np.random.gamma(self.b1,1./self.b2,(self.n_rows,self.n_components)) 176 | return np.random.poisson(np.inner(latent_a,latent_b)) 177 | 178 | 179 | 180 | def _compute_expectations(alpha, beta): 181 | ''' 182 | Given x ~ Gam(alpha, beta), compute E[x] and E[log x] 183 | ''' 184 | #beta=beta.reshape((beta.shape[0], 1)) 185 | return (alpha / beta, special.psi(alpha) - np.log(beta)) 186 | 187 | def _compute_entropy(alpha, beta): 188 | ''' 189 | Given x ~ Gam(alpha, beta), compute Entropy[x] 190 | ''' 191 | #beta=beta.reshape((beta.shape[0], 1)) 192 | return alpha+(1-alpha)*special.psi(alpha) - np.log(beta)+special.gammaln(alpha) 193 | 194 | def _gamma_term(a, b, shape, rate, Ex, Elogx): 195 | return np.sum((a - shape) * Elogx - (b - rate) * Ex + 196 | (special.gammaln(shape) - shape * np.log(rate))) 197 | 198 | def _sum_product_newaxis1(auxvar, data, axis=1): 199 | return np.sum(auxvar * data[np.newaxis, :, :], axis=axis) 200 | -------------------------------------------------------------------------------- /sparse_tensor.py: -------------------------------------------------------------------------------- 1 | """ 2 | 3 | Poisson Tensor Factorization using sparse representation of input tensor by: 2017-11-24 Eliezer de Souza da Silva 4 | Modification of a code created in: 2014-03-25 02:06:52 by Dawen Liang 5 | 6 | """ 7 | 8 | import sys 9 | import numpy as np 10 | from scipy import special 11 | import numpy_indexed as npi 12 | 13 | 14 | from sklearn.base import BaseEstimator, TransformerMixin 15 | 16 | 17 | class PoissonTF(BaseEstimator, TransformerMixin): 18 | """ Poisson tensor factorization with batch inference """ 19 | def __init__(self, n_components=100, max_iter=100, tol=0.0005, 20 | smoothness=100, random_state=None, verbose=False, 21 | **kwargs): 22 | """ Poisson tensor factorization 23 | 24 | Arguments 25 | --------- 26 | n_components : int 27 | Number of latent components 28 | 29 | max_iter : int 30 | Maximal number of iterations to perform 31 | 32 | tol : float 33 | The threshold on the increase of the objective to stop the 34 | iteration 35 | 36 | smoothness : int 37 | Smoothness on the initialization variational parameters 38 | 39 | random_state : int or RandomState 40 | Pseudo random number generator used for sampling 41 | 42 | verbose : bool 43 | Whether to show progress during model fitting 44 | 45 | **kwargs: dict 46 | Model hyperparameters 47 | """ 48 | 49 | self.n_components = n_components 50 | self.max_iter = max_iter 51 | self.tol = tol 52 | self.smoothness = smoothness 53 | self.random_state = random_state 54 | self.verbose = verbose 55 | 56 | if type(self.random_state) is int: 57 | np.random.seed(self.random_state) 58 | elif self.random_state is not None: 59 | np.random.setstate(self.random_state) 60 | 61 | self._parse_args(**kwargs) 62 | 63 | def _parse_args(self, **kwargs): 64 | self.a = float(kwargs.get('a', 0.1)) 65 | self.b = float(kwargs.get('b', 0.1)) 66 | 67 | def _init_components(self, n_rows): 68 | # variational parameters modes 69 | self.mode_sizes = n_rows 70 | self.gamma_b=[] 71 | self.rho_b=[] 72 | self.Eb=[] 73 | self.Elogb=[] 74 | self.partial_sums=[] 75 | self.gamma_lambda=np.random.gamma(self.smoothness, 1. / self.smoothness, 76 | size=(1, self.n_components)) 77 | self.rho_lambda=np.random.gamma(self.smoothness, 1. / self.smoothness, 78 | size=(1, self.n_components)) 79 | self.Elambda,self.Eloglambda=_compute_expectations(self.gamma_lambda, self.rho_lambda) 80 | for mode_n in n_rows: 81 | gamma_b=self.smoothness \ 82 | * np.random.gamma(self.smoothness, 1. / self.smoothness, 83 | size=(mode_n, self.n_components)) 84 | rho_b = self.smoothness \ 85 | * np.random.gamma(self.smoothness, 1. / self.smoothness, 86 | size=(mode_n, self.n_components)) 87 | self.gamma_b.append(gamma_b) 88 | self.rho_b.append(rho_b) 89 | tempE, tempElog = _compute_expectations(gamma_b, rho_b) 90 | self.partial_sums.append(np.sum(tempE,axis=0,keepdims=True)) ### shape=(1,self.n_components) 91 | self.Eb.append(tempE) 92 | self.Elogb.append(tempElog) 93 | 94 | def set_components(self, shape, rate): 95 | '''Set the latent components from variational parameters. 96 | 97 | Parameters 98 | ---------- 99 | shape : list of numpy-array, shape (n_items_mode, n_components) 100 | Shape parameters for the variational distribution 101 | 102 | rate : list of numpy-array, shape (n_items_mode, n_components) 103 | Rate parameters for the variational distribution 104 | 105 | Returns 106 | ------- 107 | self : object 108 | Return the instance itself. 109 | ''' 110 | 111 | self.gamma_b, self.rho_b = shape, rate 112 | self.Eb, self.Elogb = shape,rate 113 | for s,r,i in zip(shape,rate,range(len(shape))): 114 | self.Eb[i], self.Elogb[i] = _compute_expectations(s, r) 115 | return self 116 | 117 | def fit(self, X): 118 | '''Fit the model to the data in X. 119 | 120 | Parameters 121 | ---------- 122 | X : sparse tensor array-like, shape (n_examples, n_modes+1) 123 | 124 | Training data. 125 | [[X_mode_1,...,X_mode_n,V], 126 | ....] 127 | 128 | V[X_mode_1,...,X_mode]] is the value indexed by [X_mode_1,...,X_mode_n] in a dense tensor array 129 | 130 | Returns 131 | ------- 132 | self: object 133 | Returns the instance itself. 134 | ''' 135 | self.n_ids_mode = [] 136 | self.unique_id_mode=[] 137 | X_new=np.zeros(shape=X.shape,dtype=int) 138 | X_new[:, -1]=X[:,-1] ## copy the last column 139 | for mode in range(X.shape[1]-1): ## the last column is the value, is not a mode 140 | unique_ids= np.unique(X[:,mode]) 141 | self.unique_id_mode.append(unique_ids) 142 | d_ids = dict(zip(unique_ids,range(len(unique_ids)))) 143 | X_new[:, mode] = np.array([d_ids[x] for x in X[:, mode]]) 144 | self.n_ids_mode.append( np.max(X_new[:,mode])+1 ) 145 | if self.verbose: 146 | print("n_ids in mode "+str(mode)+"= "+str(self.n_ids_mode[mode])) 147 | self._init_components(self.n_ids_mode) 148 | self._update(X_new) 149 | return self 150 | 151 | def transform(self, X, attr=None): 152 | '''Encode the data as a linear combination of the latent components. 153 | 154 | Parameters 155 | ---------- 156 | X : array-like, shape (n_samples, n_feats) 157 | 158 | attr: string 159 | The name of attribute, default 'Eb'. Can be changed to Elogb to 160 | obtain E_q[log beta] as transformed data. 161 | 162 | Returns 163 | ------- 164 | X_new : array-like, shape(n_samples, n_filters) 165 | Transformed data, as specified by attr. 166 | ''' 167 | ''' 168 | 169 | if not hasattr(self, 'Eb'): 170 | raise ValueError('There are no pre-trained components.') 171 | n_samples, n_feats = X.shape 172 | if n_feats != self.Eb.shape[1]: 173 | raise ValueError('The dimension of the transformed data ' 174 | 'does not match with the existing components.') 175 | if attr is None: 176 | attr = 'Et' 177 | self._init_weights(n_samples) 178 | self._update(X, update_beta=False) 179 | return getattr(self, attr) 180 | ''' 181 | def _update(self, X, update_beta=True): 182 | # alternating between update latent components and weights 183 | old_bd = -np.inf 184 | self._update_phi(X) 185 | 186 | for i in xrange(self.max_iter): 187 | self._update_lambda(X) 188 | self._update_latent_factors(X) 189 | self._update_phi(X) 190 | bound = self._bound(X) 191 | improvement = (bound - old_bd) / abs(old_bd) 192 | if self.verbose: 193 | sys.stdout.write('\r\tAfter ITERATION: %d\tObjective: %.0f\t' 194 | 'Old objective: %.0f\t' 195 | 'Improvement: %.5f' % (i, bound, old_bd, 196 | improvement)) 197 | sys.stdout.flush() 198 | if np.abs(improvement) < self.tol: 199 | break 200 | old_bd = bound 201 | if self.verbose: 202 | sys.stdout.write('\n') 203 | pass 204 | def _update_lambda(self,X): 205 | self.gamma_lambda = self.a+self.phi_var_data.sum(axis=0,keepdims=True) 206 | self.rho_lambda = self.b 207 | for mode in range(X.shape[1]-1): 208 | self.rho_lambda+=self.Eb[mode].sum(axis=0,keepdims=True) 209 | self.Elambda,self.Eloglambda=_compute_expectations(self.gamma_lambda, self.rho_lambda) 210 | 211 | 212 | def _update_phi(self,X): 213 | self.phi_var = np.zeros((X.shape[0], self.n_components)) ### start zeroing everything 214 | for mode in range(X.shape[1]-1): ### for each element of the mode, minus the last one, which is the value 215 | # select the non-zero evidence elements of the latent factor of each mode 216 | # [m1, m2, m3, m4, m5, v] => Data[m1,m2,m3,m5]=v, so for each mode i we select m_i = {index non zero} 217 | self.phi_var = np.add(self.phi_var, np.exp(self.Elogb[mode][X[:,mode], :])) 218 | self.phi_var = np.add(self.phi_var, np.exp(self.Eloglambda)) 219 | self.phi_var = np.divide(self.phi_var, np.sum(self.phi_var, axis=1)[:, np.newaxis]) 220 | self.phi_var_data =X[:,-1,np.newaxis]*self.phi_var ### X[m1, m2, m3, m4, m5, v][-1]=v 221 | 222 | def _update_latent_factors(self, X): 223 | # variational parameters modes 224 | for mode in range(X.shape[1]-1): 225 | self.gamma_b[mode]=self.a + npi.group_by(X[:,mode]).sum(self.phi_var_data)[1] 226 | self.rho_b[mode]=self.b + self.Elambda*np.prod(self.partial_sums[:mode]+self.partial_sums[(mode+1):], axis=0) 227 | tempE, tempElog = _compute_expectations(self.gamma_b[mode], self.rho_b[mode]) 228 | self.partial_sums[mode]=np.sum(tempE,axis=0,keepdims=True) ### shape=(1,self.n_components) 229 | self.Eb[mode]=tempE 230 | self.Elogb[mode]=tempElog 231 | 232 | def _xexplog(self): 233 | ''' 234 | sum_k exp(E[log theta_{ik} * beta_{kd}]) 235 | ''' 236 | return np.dot(np.exp(self.Elogt), np.exp(self.Elogb)) 237 | 238 | def _bound(self, X): 239 | bound=0 240 | bound+=np.sum(self.phi_var_data*self.Eloglambda) 241 | for mode in range(X.shape[1]-1): 242 | bound+=np.sum(self.phi_var_data*self.Elogb[mode][X[:,mode], :]) 243 | bound-=np.sum(self.Elambda*np.prod(self.partial_sums,axis=0)) 244 | bound += _gamma_term(self.a, self.a , 245 | self.gamma_b[mode], self.rho_b[mode], 246 | self.Eb[mode], self.Elogb[mode]) 247 | bound += _gamma_term(self.a, self.a , 248 | self.gamma_lambda, self.rho_lambda, 249 | self.Elambda, self.Eloglambda) 250 | bound -= np.sum(np.log(self.phi_var+0.00000000000001)*self.phi_var_data) 251 | 252 | 253 | 254 | return bound 255 | 256 | 257 | def _compute_expectations(alpha, beta): 258 | ''' 259 | Given x ~ Gam(alpha, beta), compute E[x] and E[log x] 260 | ''' 261 | #beta=beta.reshape((beta.shape[0], 1)) 262 | # TODO: maybe use 1-dimensional beta and broadcast in the appropriate dimension... save memory if it is a problem 263 | return (alpha / beta, special.psi(alpha) - np.log(beta)) 264 | 265 | 266 | def _gamma_term(a, b, shape, rate, Ex, Elogx): 267 | return np.sum((a - shape) * Elogx - (b - rate) * Ex + 268 | (special.gammaln(shape) - shape * np.log(rate))) 269 | 270 | 271 | def _sum_product_newaxis1(auxvar, data, axis=1): 272 | return np.sum(auxvar * data[np.newaxis, :, :], axis=axis) 273 | 274 | def _compute_entropy(alpha, beta): 275 | ''' 276 | Given x ~ Gam(alpha, beta), compute Entropy[x] 277 | ''' 278 | #beta=beta.reshape((beta.shape[0], 1)) 279 | # TODO: use 1-dimensional beta and broadcast in the appropriate dimension... save memory if it is a problem 280 | return alpha+(1-alpha)*special.psi(alpha) - np.log(beta)+special.gammaln(alpha) --------------------------------------------------------------------------------