├── .gitignore ├── LICENSE ├── README.md ├── doc └── pymaxent.html ├── examples.ipynb ├── generate-doc.sh └── src └── pymaxent.py /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Saad Research Group @ the University of Utah 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # PyMaxEnt 2 | `PyMaxEnt` is a python software that solves the inverse moment problem using the maximum entropy principle. Given a finite number of moments, PyMaxEnt will find the function that generates those moments and also maximizes the information entropy. 3 | 4 | # Usage 5 | To use `PyMaxEnt`, a single function call to `reconstruct(moments, ivars, bnds, scaling)` is made. Here, `moments` is a required list or array of known moments, `ivars` is an optional argument containing discrete values of the independent variable, `bnds` is a tuple `[a,b]` containing the expected bounds of the resulting distribution, and `scaling` is the invariant measure or the scaling function. 6 | 7 | When `ivars` is provided, the reconstruction assumes a discrete distribution. When a discrete reconstruction is chosen, `scaling` should be an array of the same size as `ivars`. 8 | 9 | More details can be found in `src/pymaxent.py`. 10 | 11 | ## Sample code snippets 12 | 13 | Below are some examples of using the \MaxEnt software. Starting with an example to reconstruct a basic dicrete distribution with two moments and four independent variables 14 | 15 | ```python 16 | from pymaxent import * 17 | mu = [1,3.5] 18 | x = [1,2,3,4,5,6] 19 | sol, lambdas = reconstruct(mu,ivars=x) 20 | ``` 21 | 22 | Similarly, for a continuous distribution, one passes a list of input moments. 23 | In this case, however, one must specify the bounds (`bnds`) to indicate that this is a continuous reconstruction. 24 | Here's an example for a Gaussian distribution 25 | ```python 26 | from pymaxent import * 27 | mu = [1,0,0.04] 28 | sol, lambdas = reconstruct(mu,bnds=[-1,1]) 29 | # plot the reconstructed solution 30 | x = np.linspace(-1,1) 31 | plot(x,sol(x)) 32 | ``` 33 | -------------------------------------------------------------------------------- /doc/pymaxent.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | pymaxent API documentation 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 |
19 |
20 |
21 |

Module pymaxent

22 |
23 |
24 |

PyMaxEnt.py: Implements a maximum entropy reconstruction of distributions with known moments.

25 |
26 | Source code 27 | 
#!/usr/bin/env python
 28 | """PyMaxEnt.py: Implements a maximum entropy reconstruction of distributions with known moments."""
 29 | 
 30 | __author__     = "Tony Saad and Giovanna Ruai"
 31 | __copyright__  = "Copyright (c) 2019, Tony Saad"
 32 | 
 33 | __credits__    = ["University of Utah Department of Chemical Engineering", "University of Utah UROP office"]
 34 | __license__    = "MIT"
 35 | __version__    = "1.0.0"
 36 | __maintainer__ = "Tony Saad"
 37 | __email__      = "tony.saad@chemeng.utah.edu"
 38 | __status__     = "Production"
 39 | 
 40 | import numpy as np
 41 | from scipy.integrate import quad
 42 | from scipy.optimize import fsolve
 43 | 
 44 | def moments_c(f, k=0, bnds=[-np.inf, np.inf]):
 45 |     '''
 46 |     Creates "k" moments: μ0, μ1, ..., μ(k-1) for a function "f" on the support given by "bnds".
 47 | 
 48 |     Parameters:
 49 |         f (function): distribution function **must be in the form of a function**
 50 |         k (int): integer number of moments to compute. Will evaluate the first k moments of f, μ0, μ1, ..., μ(k-1)
 51 |         bnds (tuple): boundaries for the integration
 52 | 
 53 |     Returns:
 54 |         moments: an array of moments of length "k"
 55 |     
 56 |     Example:
 57 |         μ = moments(3, f, [-1, 1])    
 58 |     '''
 59 |     def mom(x, k):
 60 |         return x**k*f(x)
 61 |     
 62 |     moms = np.zeros(k)
 63 |     a = bnds[0]
 64 |     b = bnds[1]
 65 |     for i in range(0,k):
 66 |         moms[i] = quad(mom,a,b,args = i)[0]
 67 |     return moms
 68 | 
 69 | def moments_d(f,k,x):
 70 |     '''
 71 |     Calculates the first "k" moments: μ0, μ1, ..., μ(k-1) of a discrete distribution "f".
 72 |     
 73 |     Parameters:
 74 |         f (array): an array of values for a discrete distribution        
 75 |         k (int): number of moments to compute. Will evaluate the first k moments of f, μ0, μ1, ..., μ(k-1)        
 76 |         x (array): list or array containing the values of the random variable over which the distribution is to be integrated
 77 | 
 78 |     Returns:
 79 |         mom: an array of length k containing the moments for the known distribution
 80 |     '''
 81 |     moms = []
 82 |     for i in range(0,k):
 83 |         xp = np.power(x,i)      # compute x^p
 84 |         xpf = np.dot(xp,f)      # compute x^p * f(x)
 85 |         mom.append(np.sum(xpf)) # compute moment: sum(x^p * f(x))
 86 |     return np.array(moms)
 87 | 
 88 | def moments(f, k, rndvar=None, bnds=None):
 89 |     '''
 90 |     Computes the first "k" moments of a function "f" on the support given by "bnd". If "rndvar" is provided, then a discrete distribution is assumed and "f" ##must## be a list or array of scalar values.
 91 | 
 92 |     Parameters:
 93 |         f (function): distribution function **must be in the form of a function**
 94 |         k (integer): will evaluate the first k moments of f, μ0, μ1, ..., μ(k-1)
 95 |         rndvar (array): optional - designates a list or array of discrete values for a random variable. If x is provided, then the moments will be computed based on a discrete distribution. This means that f must be an array as well.
 96 |         bnds (tuple): a list of two numbers consisting of the lower and upper bounds of the support    
 97 |     
 98 |     Returns:
 99 |         moments: an array of moments of length `k`    
100 | 
101 |     Example:
102 |         μ = moments(3, f, [-1, 1])    
103 |     '''    
104 |     if rndvar is not None:
105 |         if bnds is not None:
106 |             print('WARNING: You specified BOTH x and boundaries. I will assume this is a discrete distribution. If you want to calculate a continuous distribution, please specify bnd ONLY.')
107 |         return moments_d(f,k,rndvar)
108 |     else:
109 |         return moments_c(f,k,bnds)
110 | 
111 | def integrand(x, lamb, k=0, discrete=False):
112 |     '''
113 |     Calculates the integrand of the \(k^\mathrm{th}\) moment.
114 | 
115 |     Parameters:
116 |         x (array): linear space or set of values for a random variable on which the integrand is applied
117 |         lamb (array): an array of Lagrange multipliers used to approximate the distribution
118 |         k (integer): a constant representing the order of the moment being calculated
119 | 
120 |     Returns:
121 |         integrand: the caclulated portion of the integrand at each x value
122 |     '''
123 |     neqs = len(lamb)
124 |     xi = np.array([x**i for i in range(0, neqs)])
125 |     if discrete:
126 |         return x**k * np.exp(np.dot(lamb, xi))
127 |     else:
128 |         return x**k * np.exp(np.dot(lamb, xi))
129 | 
130 | def residual_d(lamb,x,k,mu):
131 |     '''
132 |     Calculates the residual of the moment approximation function.
133 | 
134 |     Parameters:
135 |         lamb (array): an array of Lagrange constants used to approximate the distribution
136 |         x (array): 
137 |         k (integer): order of the moment        
138 |         mu (array): an array of the known moments needed to approximate the distribution function
139 |     
140 |     Returns:
141 |         rhs: the integrated right hand side of the moment approximation function
142 |     '''
143 |     l_sum = []
144 |     for i in range(0,len(lamb)):
145 |         l_sum.append( np.sum(integrand(x,lamb,i,discrete=True)) - mu[i] )
146 |     return np.array(l_sum)
147 | 
148 | def maxent_reconstruct_d(rndvar, mu):
149 |     '''
150 |     Computes the most likely distribution from the moments given using maximum entropy theorum.
151 | 
152 |     Parameters:
153 |         rndvar (array): a list or array of known dependent variables. For example, for a 6-faced die, rndvar=[1,2,3,4,5,6]
154 |         mu (array): vector of size m containing the known moments of a distribution. This does NOT assume that μ0 = 1. This vector contains moments μ_k starting with μ_0, μ_1, etc... For example, μ = [1,0,0]
155 | 
156 |     Returns:
157 |         probabilites: vector of size b (from bnd[1]) containing the probabilities for the distribution 
158 |         lambsol: vector of lagrangian multipliers
159 |     '''
160 |     lambguess = np.zeros(len(mu))
161 |     lambguess[0] = -np.log(np.sqrt(2*np.pi))
162 |     k = len(mu)
163 |     lambsol = fsolve(residual_d, lambguess, args = (rndvar,k,mu))
164 |     probabilites = integrand(rndvar, lambsol, k=0, discrete=True)    
165 |     return probabilites, lambsol
166 | 
167 | 
168 | def residual_c(lamb, mu, bnds):
169 |     '''
170 |     Calculates the residual of the moment approximation function.
171 |     
172 |     Parameters:
173 |         lamb (array): an array of Lagrange constants used to approximate the distribution
174 |         mu (array): an array of the known moments needed to approximate the distribution function
175 |         bnds (tuple): support bounds
176 | 
177 |     Returns:
178 |         rhs: the integrated right hand side of the moment approximation function
179 |     '''
180 |     a = bnds[0]
181 |     b = bnds[1]
182 |     neqs = len(lamb)
183 |     rhs = np.zeros(neqs)
184 |     for k in range(0, neqs):
185 |         rhs[k] = quad(integrand, a, b, args=(lamb, k))[0] - mu[k]
186 |     return rhs
187 | 
188 | def maxent_reconstruct_c(mu, bnds=[-np.inf, np.inf]):
189 |     '''
190 |     Used to construct a continuous distribution from a limited number of known moments(μ). This function applies Maximum Entropy Theory in order to solve for the constraints found in the approximation equation that is given as an output.
191 |     
192 |     Parameters:
193 |         μ: vector of size m containing the known moments of a distribution. This does NOT assume that μ0 = 1. This vector contains moments μ_k starting with μ_0, μ_1, etc...
194 |             Ex. μ = [1,0,0]
195 |         bnds: Support for the integration [a,b]
196 |             ## It is important the bounds include roughly all non-zero values of the distribution that is being recreated ##
197 |     
198 |     Returns:
199 |         Distribution Function: The recreated probability distribution function from the moment vector (μ) input given. requires a support to be ploted
200 |     
201 |     Example:
202 |         >>> f, sol = maxent([1,0,0], [-1,1])        
203 |     '''
204 |     neqs = len(mu)
205 |     lambguess = np.zeros(neqs) # initialize guesses
206 |     lambguess[0] = -np.log(np.sqrt(2*np.pi)) # set the first initial guess - this seems to work okay
207 |     lambsol = fsolve(residual_c, lambguess, args=(mu,bnds), col_deriv=True)
208 |     recon = lambda x: integrand(x, lambsol, k=0)
209 |     return recon, lambsol
210 | 
211 | def reconstruct(mu, rndvar=None, bnds=None):
212 |     '''
213 |     This is the main function call to generate maximum entropy solutions.
214 |     
215 |     Parameters:
216 |         mu (array): a list or array of known moments
217 |         rndvar (array): optional - a list or array of known dependent variables. For example, for a 6-faced die, rndvar=[1,2,3,4,5,6]. If rndvar is provided, we will assume a discrete reconstruction.
218 |         bnds (tuple): a tuple [a,b] containing the bounds or support of the reconstructed solution. This is only required for continuous distributions and will be neglected if rndvar is provided.
219 |     
220 |     Returns:
221 |         recon: reconstructed distribution. If continuous, then `recon` is a Python function, `f(x)`. If discrete, then recon is an array of probabilities.
222 |         lambsol (array): array containing the lagrangian multipliers
223 |     
224 |     Examples:
225 |         ### reconstruct a discrete distribution
226 |         >>> from pymaxent import *
227 |         >>> mu = [1,3.5]
228 |         >>> x = [1,2,3,4,5,6]
229 |         >>> sol, lambdas = reconstruct(mu,rndvar=x)
230 |         
231 |         ### reconstruct a continuous distribution
232 |         >>> from pymaxent import *
233 |         >>> mu = [1,0,0.04]
234 |         >>> sol, lambdas = reconstruct(mu,bnds=[-1,1])
235 |         >>> x = np.linspace(-1,1)
236 |         >>> plot(x,sol(x))              
237 |     '''
238 |     result = 0
239 |     # Discrete case
240 |     if rndvar is not None:
241 |         rndvar = np.array(rndvar) # convert things to numpy arrays
242 |         if bnds is not None:
243 |             print('WARNING: You specified BOTH x and boundaries. I will assume this is a discrete distribution. If you want to calculate a continuous distribution, please specify bnd ONLY.')
244 |         result = maxent_reconstruct_d(rndvar, mu)
245 |     # Continuous case
246 |     else:
247 |         result = maxent_reconstruct_c(mu, bnds)
248 |     return result
249 |
250 |
251 |
252 |
253 |
254 |
255 |
256 |

Functions

257 |
258 |
259 | def integrand(x, lamb, k=0, discrete=False) 260 |
261 |
262 |

Calculates the integrand of the k^\mathrm{th} moment.

263 |

Parameters

264 |
265 |
x : array
266 |
linear space or set of values for a random variable on which the integrand is applied
267 |
lamb : array
268 |
an array of Lagrange multipliers used to approximate the distribution
269 |
k : integer
270 |
a constant representing the order of the moment being calculated
271 |
272 |

Returns

273 |
274 |
integrand()
275 |
the caclulated portion of the integrand at each x value
276 |
277 |
278 | Source code 279 | 
def integrand(x, lamb, k=0, discrete=False):
280 |     '''
281 |     Calculates the integrand of the \(k^\mathrm{th}\) moment.
282 | 
283 |     Parameters:
284 |         x (array): linear space or set of values for a random variable on which the integrand is applied
285 |         lamb (array): an array of Lagrange multipliers used to approximate the distribution
286 |         k (integer): a constant representing the order of the moment being calculated
287 | 
288 |     Returns:
289 |         integrand: the caclulated portion of the integrand at each x value
290 |     '''
291 |     neqs = len(lamb)
292 |     xi = np.array([x**i for i in range(0, neqs)])
293 |     if discrete:
294 |         return x**k * np.exp(np.dot(lamb, xi))
295 |     else:
296 |         return x**k * np.exp(np.dot(lamb, xi))
297 |
298 |
299 |
300 | def maxent_reconstruct_c(mu, bnds=[-inf, inf]) 301 |
302 |
303 |

Used to construct a continuous distribution from a limited number of known moments(μ). This function applies Maximum Entropy Theory in order to solve for the constraints found in the approximation equation that is given as an output.

304 |

Parameters

305 |
306 |
μ
307 |
vector of size m containing the known moments of a distribution. This does NOT assume that μ0 = 1. This vector contains moments μ_k starting with μ_0, μ_1, etc… 308 | Ex. μ = [1,0,0]
309 |
bnds
310 |
Support for the integration [a,b]

It is important the bounds include roughly all non-zero values of the distribution that is being recreated

311 |
312 |
313 |

Returns

314 |
315 |
Distribution Function: The recreated probability distribution function from the moment vector (μ) input given. requires a support to be ploted
316 |
 
317 |
318 |

Example

319 | 
>>> f, sol = maxent([1,0,0], [-1,1])
320 |
321 |
322 | Source code 323 | 
def maxent_reconstruct_c(mu, bnds=[-np.inf, np.inf]):
324 |     '''
325 |     Used to construct a continuous distribution from a limited number of known moments(μ). This function applies Maximum Entropy Theory in order to solve for the constraints found in the approximation equation that is given as an output.
326 |     
327 |     Parameters:
328 |         μ: vector of size m containing the known moments of a distribution. This does NOT assume that μ0 = 1. This vector contains moments μ_k starting with μ_0, μ_1, etc...
329 |             Ex. μ = [1,0,0]
330 |         bnds: Support for the integration [a,b]
331 |             ## It is important the bounds include roughly all non-zero values of the distribution that is being recreated ##
332 |     
333 |     Returns:
334 |         Distribution Function: The recreated probability distribution function from the moment vector (μ) input given. requires a support to be ploted
335 |     
336 |     Example:
337 |         >>> f, sol = maxent([1,0,0], [-1,1])        
338 |     '''
339 |     neqs = len(mu)
340 |     lambguess = np.zeros(neqs) # initialize guesses
341 |     lambguess[0] = -np.log(np.sqrt(2*np.pi)) # set the first initial guess - this seems to work okay
342 |     lambsol = fsolve(residual_c, lambguess, args=(mu,bnds), col_deriv=True)
343 |     recon = lambda x: integrand(x, lambsol, k=0)
344 |     return recon, lambsol
345 |
346 |
347 |
348 | def maxent_reconstruct_d(rndvar, mu) 349 |
350 |
351 |

Computes the most likely distribution from the moments given using maximum entropy theorum.

352 |

Parameters

353 |
354 |
rndvar : array
355 |
a list or array of known dependent variables. For example, for a 6-faced die, rndvar=[1,2,3,4,5,6]
356 |
mu : array
357 |
vector of size m containing the known moments of a distribution. This does NOT assume that μ0 = 1. This vector contains moments μ_k starting with μ_0, μ_1, etc… For example, μ = [1,0,0]
358 |
359 |

Returns

360 |
361 |
probabilites
362 |
vector of size b (from bnd[1]) containing the probabilities for the distribution
363 |
lambsol
364 |
vector of lagrangian multipliers
365 |
366 |
367 | Source code 368 | 
def maxent_reconstruct_d(rndvar, mu):
369 |     '''
370 |     Computes the most likely distribution from the moments given using maximum entropy theorum.
371 | 
372 |     Parameters:
373 |         rndvar (array): a list or array of known dependent variables. For example, for a 6-faced die, rndvar=[1,2,3,4,5,6]
374 |         mu (array): vector of size m containing the known moments of a distribution. This does NOT assume that μ0 = 1. This vector contains moments μ_k starting with μ_0, μ_1, etc... For example, μ = [1,0,0]
375 | 
376 |     Returns:
377 |         probabilites: vector of size b (from bnd[1]) containing the probabilities for the distribution 
378 |         lambsol: vector of lagrangian multipliers
379 |     '''
380 |     lambguess = np.zeros(len(mu))
381 |     lambguess[0] = -np.log(np.sqrt(2*np.pi))
382 |     k = len(mu)
383 |     lambsol = fsolve(residual_d, lambguess, args = (rndvar,k,mu))
384 |     probabilites = integrand(rndvar, lambsol, k=0, discrete=True)    
385 |     return probabilites, lambsol
386 |
387 |
388 |
389 | def moments(f, k, rndvar=None, bnds=None) 390 |
391 |
392 |

Computes the first "k" moments of a function "f" on the support given by "bnd". If "rndvar" is provided, then a discrete distribution is assumed and "f" ##must## be a list or array of scalar values.

393 |

Parameters

394 |
395 |
f : function
396 |
distribution function must be in the form of a function
397 |
k : integer
398 |
will evaluate the first k moments of f, μ0, μ1, …, μ(k-1)
399 |
rndvar : array
400 |
optional - designates a list or array of discrete values for a random variable. If x is provided, then the moments will be computed based on a discrete distribution. This means that f must be an array as well.
401 |
bnds : tuple
402 |
a list of two numbers consisting of the lower and upper bounds of the support 403 |
404 |
405 |

Returns

406 |
407 |
moments()
408 |
an array of moments of length k 409 |
410 |
411 |

Example

412 |

μ = moments(3, f, [-1, 1])

413 |
414 | Source code 415 | 
def moments(f, k, rndvar=None, bnds=None):
416 |     '''
417 |     Computes the first "k" moments of a function "f" on the support given by "bnd". If "rndvar" is provided, then a discrete distribution is assumed and "f" ##must## be a list or array of scalar values.
418 | 
419 |     Parameters:
420 |         f (function): distribution function **must be in the form of a function**
421 |         k (integer): will evaluate the first k moments of f, μ0, μ1, ..., μ(k-1)
422 |         rndvar (array): optional - designates a list or array of discrete values for a random variable. If x is provided, then the moments will be computed based on a discrete distribution. This means that f must be an array as well.
423 |         bnds (tuple): a list of two numbers consisting of the lower and upper bounds of the support    
424 |     
425 |     Returns:
426 |         moments: an array of moments of length `k`    
427 | 
428 |     Example:
429 |         μ = moments(3, f, [-1, 1])    
430 |     '''    
431 |     if rndvar is not None:
432 |         if bnds is not None:
433 |             print('WARNING: You specified BOTH x and boundaries. I will assume this is a discrete distribution. If you want to calculate a continuous distribution, please specify bnd ONLY.')
434 |         return moments_d(f,k,rndvar)
435 |     else:
436 |         return moments_c(f,k,bnds)
437 |
438 |
439 |
440 | def moments_c(f, k=0, bnds=[-inf, inf]) 441 |
442 |
443 |

Creates "k" moments: μ0, μ1, …, μ(k-1) for a function "f" on the support given by "bnds".

444 |

Parameters

445 |
446 |
f : function
447 |
distribution function must be in the form of a function
448 |
k : int
449 |
integer number of moments to compute. Will evaluate the first k moments of f, μ0, μ1, …, μ(k-1)
450 |
bnds : tuple
451 |
boundaries for the integration
452 |
453 |

Returns

454 |
455 |
moments()
456 |
an array of moments of length "k"
457 |
458 |

Example

459 |

μ = moments(3, f, [-1, 1])

460 |
461 | Source code 462 | 
def moments_c(f, k=0, bnds=[-np.inf, np.inf]):
463 |     '''
464 |     Creates "k" moments: μ0, μ1, ..., μ(k-1) for a function "f" on the support given by "bnds".
465 | 
466 |     Parameters:
467 |         f (function): distribution function **must be in the form of a function**
468 |         k (int): integer number of moments to compute. Will evaluate the first k moments of f, μ0, μ1, ..., μ(k-1)
469 |         bnds (tuple): boundaries for the integration
470 | 
471 |     Returns:
472 |         moments: an array of moments of length "k"
473 |     
474 |     Example:
475 |         μ = moments(3, f, [-1, 1])    
476 |     '''
477 |     def mom(x, k):
478 |         return x**k*f(x)
479 |     
480 |     moms = np.zeros(k)
481 |     a = bnds[0]
482 |     b = bnds[1]
483 |     for i in range(0,k):
484 |         moms[i] = quad(mom,a,b,args = i)[0]
485 |     return moms
486 |
487 |
488 |
489 | def moments_d(f, k, x) 490 |
491 |
492 |

Calculates the first "k" moments: μ0, μ1, …, μ(k-1) of a discrete distribution "f".

493 |

Parameters

494 |
495 |
f : array
496 |
an array of values for a discrete distribution 497 |
498 |
k : int
499 |
number of moments to compute. Will evaluate the first k moments of f, μ0, μ1, …, μ(k-1) 500 |
501 |
x : array
502 |
list or array containing the values of the random variable over which the distribution is to be integrated
503 |
504 |

Returns

505 |
506 |
mom
507 |
an array of length k containing the moments for the known distribution
508 |
509 |
510 | Source code 511 | 
def moments_d(f,k,x):
512 |     '''
513 |     Calculates the first "k" moments: μ0, μ1, ..., μ(k-1) of a discrete distribution "f".
514 |     
515 |     Parameters:
516 |         f (array): an array of values for a discrete distribution        
517 |         k (int): number of moments to compute. Will evaluate the first k moments of f, μ0, μ1, ..., μ(k-1)        
518 |         x (array): list or array containing the values of the random variable over which the distribution is to be integrated
519 | 
520 |     Returns:
521 |         mom: an array of length k containing the moments for the known distribution
522 |     '''
523 |     moms = []
524 |     for i in range(0,k):
525 |         xp = np.power(x,i)      # compute x^p
526 |         xpf = np.dot(xp,f)      # compute x^p * f(x)
527 |         mom.append(np.sum(xpf)) # compute moment: sum(x^p * f(x))
528 |     return np.array(moms)
529 |
530 |
531 |
532 | def reconstruct(mu, rndvar=None, bnds=None) 533 |
534 |
535 |

This is the main function call to generate maximum entropy solutions.

536 |

Parameters

537 |
538 |
mu : array
539 |
a list or array of known moments
540 |
rndvar : array
541 |
optional - a list or array of known dependent variables. For example, for a 6-faced die, rndvar=[1,2,3,4,5,6]. If rndvar is provided, we will assume a discrete reconstruction.
542 |
bnds : tuple
543 |
a tuple [a,b] containing the bounds or support of the reconstructed solution. This is only required for continuous distributions and will be neglected if rndvar is provided.
544 |
545 |

Returns

546 |
547 |
recon
548 |
reconstructed distribution. If continuous, then recon is a Python function, f(x). If discrete, then recon is an array of probabilities.
549 |
lambsol : array
550 |
array containing the lagrangian multipliers
551 |
552 |

Examples

553 |

reconstruct a discrete distribution

554 | 
>>> from pymaxent import *
555 | >>> mu = [1,3.5]
556 | >>> x = [1,2,3,4,5,6]
557 | >>> sol, lambdas = reconstruct(mu,rndvar=x)
558 |
559 |

reconstruct a continuous distribution

560 | 
>>> from pymaxent import *
561 | >>> mu = [1,0,0.04]
562 | >>> sol, lambdas = reconstruct(mu,bnds=[-1,1])
563 | >>> x = np.linspace(-1,1)
564 | >>> plot(x,sol(x))
565 |
566 |
567 | Source code 568 | 
def reconstruct(mu, rndvar=None, bnds=None):
569 |     '''
570 |     This is the main function call to generate maximum entropy solutions.
571 |     
572 |     Parameters:
573 |         mu (array): a list or array of known moments
574 |         rndvar (array): optional - a list or array of known dependent variables. For example, for a 6-faced die, rndvar=[1,2,3,4,5,6]. If rndvar is provided, we will assume a discrete reconstruction.
575 |         bnds (tuple): a tuple [a,b] containing the bounds or support of the reconstructed solution. This is only required for continuous distributions and will be neglected if rndvar is provided.
576 |     
577 |     Returns:
578 |         recon: reconstructed distribution. If continuous, then `recon` is a Python function, `f(x)`. If discrete, then recon is an array of probabilities.
579 |         lambsol (array): array containing the lagrangian multipliers
580 |     
581 |     Examples:
582 |         ### reconstruct a discrete distribution
583 |         >>> from pymaxent import *
584 |         >>> mu = [1,3.5]
585 |         >>> x = [1,2,3,4,5,6]
586 |         >>> sol, lambdas = reconstruct(mu,rndvar=x)
587 |         
588 |         ### reconstruct a continuous distribution
589 |         >>> from pymaxent import *
590 |         >>> mu = [1,0,0.04]
591 |         >>> sol, lambdas = reconstruct(mu,bnds=[-1,1])
592 |         >>> x = np.linspace(-1,1)
593 |         >>> plot(x,sol(x))              
594 |     '''
595 |     result = 0
596 |     # Discrete case
597 |     if rndvar is not None:
598 |         rndvar = np.array(rndvar) # convert things to numpy arrays
599 |         if bnds is not None:
600 |             print('WARNING: You specified BOTH x and boundaries. I will assume this is a discrete distribution. If you want to calculate a continuous distribution, please specify bnd ONLY.')
601 |         result = maxent_reconstruct_d(rndvar, mu)
602 |     # Continuous case
603 |     else:
604 |         result = maxent_reconstruct_c(mu, bnds)
605 |     return result
606 |
607 |
608 |
609 | def residual_c(lamb, mu, bnds) 610 |
611 |
612 |

Calculates the residual of the moment approximation function.

613 |

Parameters

614 |
615 |
lamb : array
616 |
an array of Lagrange constants used to approximate the distribution
617 |
mu : array
618 |
an array of the known moments needed to approximate the distribution function
619 |
bnds : tuple
620 |
support bounds
621 |
622 |

Returns

623 |
624 |
rhs
625 |
the integrated right hand side of the moment approximation function
626 |
627 |
628 | Source code 629 | 
def residual_c(lamb, mu, bnds):
630 |     '''
631 |     Calculates the residual of the moment approximation function.
632 |     
633 |     Parameters:
634 |         lamb (array): an array of Lagrange constants used to approximate the distribution
635 |         mu (array): an array of the known moments needed to approximate the distribution function
636 |         bnds (tuple): support bounds
637 | 
638 |     Returns:
639 |         rhs: the integrated right hand side of the moment approximation function
640 |     '''
641 |     a = bnds[0]
642 |     b = bnds[1]
643 |     neqs = len(lamb)
644 |     rhs = np.zeros(neqs)
645 |     for k in range(0, neqs):
646 |         rhs[k] = quad(integrand, a, b, args=(lamb, k))[0] - mu[k]
647 |     return rhs
648 |
649 |
650 |
651 | def residual_d(lamb, x, k, mu) 652 |
653 |
654 |

Calculates the residual of the moment approximation function.

655 |

Parameters

656 |
657 |
lamb : array
658 |
an array of Lagrange constants used to approximate the distribution
659 |
x : array
660 |
 
661 |
k : integer
662 |
order of the moment 663 |
664 |
mu : array
665 |
an array of the known moments needed to approximate the distribution function
666 |
667 |

Returns

668 |
669 |
rhs
670 |
the integrated right hand side of the moment approximation function
671 |
672 |
673 | Source code 674 | 
def residual_d(lamb,x,k,mu):
675 |     '''
676 |     Calculates the residual of the moment approximation function.
677 | 
678 |     Parameters:
679 |         lamb (array): an array of Lagrange constants used to approximate the distribution
680 |         x (array): 
681 |         k (integer): order of the moment        
682 |         mu (array): an array of the known moments needed to approximate the distribution function
683 |     
684 |     Returns:
685 |         rhs: the integrated right hand side of the moment approximation function
686 |     '''
687 |     l_sum = []
688 |     for i in range(0,len(lamb)):
689 |         l_sum.append( np.sum(integrand(x,lamb,i,discrete=True)) - mu[i] )
690 |     return np.array(l_sum)
691 |
692 |
693 |
694 |
695 |
696 |
697 |
698 | 719 |
720 | 723 | 724 | 725 | 726 | -------------------------------------------------------------------------------- /generate-doc.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | dir=doc 3 | if [ -d $dir ] 4 | then 5 | pdoc --html src/pymaxent.py --output-dir doc -f 6 | else 7 | pdoc --html src/pymaxent.py --output-dir doc 8 | fi -------------------------------------------------------------------------------- /src/pymaxent.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | """PyMaxEnt.py: Implements a maximum entropy reconstruction of distributions with known moments.""" 3 | 4 | __author__ = "Tony Saad and Giovanna Ruai" 5 | __copyright__ = "Copyright (c) 2019, Tony Saad" 6 | 7 | __credits__ = ["University of Utah Department of Chemical Engineering", "University of Utah UROP office"] 8 | __license__ = "MIT" 9 | __version__ = "1.0.0" 10 | __maintainer__ = "Tony Saad" 11 | __email__ = "tony.saad@chemeng.utah.edu" 12 | __status__ = "Production" 13 | 14 | import numpy as np 15 | from scipy.integrate import quad 16 | from scipy.optimize import fsolve 17 | 18 | def moments_c(f, k=0, bnds=[-np.inf, np.inf]): 19 | ''' 20 | Creates "k" moments: μ0, μ1, ..., μ(k-1) for a function "f" on the support given by "bnds". 21 | 22 | Parameters: 23 | f (function): distribution function **must be in the form of a function** 24 | k (int): integer number of moments to compute. Will evaluate the first k moments of f, μ0, μ1, ..., μ(k-1) 25 | bnds (tuple): boundaries for the integration 26 | 27 | Returns: 28 | moments: an array of moments of length "k" 29 | 30 | Example: 31 | μ = moments(3, f, [-1, 1]) 32 | ''' 33 | def mom(x, k): 34 | return x**k*f(x) 35 | 36 | moms = np.zeros(k) 37 | a = bnds[0] 38 | b = bnds[1] 39 | for i in range(0,k): 40 | moms[i] = quad(mom,a,b,args = i)[0] 41 | return moms 42 | 43 | def moments_d(f,k,x): 44 | ''' 45 | Calculates the first "k" moments: μ0, μ1, ..., μ(k-1) of a discrete distribution "f". 46 | 47 | Parameters: 48 | f (array): an array of values for a discrete distribution 49 | k (int): number of moments to compute. Will evaluate the first k moments of f, μ0, μ1, ..., μ(k-1) 50 | x (array): list or array containing the values of the random variable over which the distribution is to be integrated 51 | 52 | Returns: 53 | mom: an array of length k containing the moments for the known distribution 54 | ''' 55 | moms = [] 56 | for i in range(0,k): 57 | xp = np.power(x,i) # compute x^p 58 | xpf = np.dot(xp,f) # compute x^p * f(x) 59 | mom.append(np.sum(xpf)) # compute moment: sum(x^p * f(x)) 60 | return np.array(moms) 61 | 62 | def moments(f, k, rndvar=None, bnds=None): 63 | ''' 64 | Computes the first "k" moments of a function "f" on the support given by "bnd". If "rndvar" is provided, then a discrete distribution is assumed and "f" ##must## be a list or array of scalar values. 65 | 66 | Parameters: 67 | f (function): distribution function **must be in the form of a function** 68 | k (integer): will evaluate the first k moments of f, μ0, μ1, ..., μ(k-1) 69 | rndvar (array): optional - designates a list or array of discrete values for a random variable. If x is provided, then the moments will be computed based on a discrete distribution. This means that f must be an array as well. 70 | bnds (tuple): a list of two numbers consisting of the lower and upper bounds of the support 71 | 72 | Returns: 73 | moments: an array of moments of length `k` 74 | 75 | Example: 76 | μ = moments(3, f, [-1, 1]) 77 | ''' 78 | if rndvar is not None: 79 | if bnds is not None: 80 | print('WARNING: You specified BOTH x and boundaries. I will assume this is a discrete distribution. If you want to calculate a continuous distribution, please specify bnd ONLY.') 81 | return moments_d(f,k,rndvar) 82 | else: 83 | return moments_c(f,k,bnds) 84 | 85 | def integrand(x, lamb, k=0, discrete=False): 86 | ''' 87 | Calculates the integrand of the \(k^\mathrm{th}\) moment. 88 | 89 | Parameters: 90 | x (array): linear space or set of values for a random variable on which the integrand is applied 91 | lamb (array): an array of Lagrange multipliers used to approximate the distribution 92 | k (integer): a constant representing the order of the moment being calculated 93 | 94 | Returns: 95 | integrand: the caclulated portion of the integrand at each x value 96 | ''' 97 | neqs = len(lamb) 98 | xi = np.array([x**i for i in range(0, neqs)]) 99 | if discrete: 100 | return x**k * np.exp(np.dot(lamb, xi)) 101 | else: 102 | return x**k * np.exp(np.dot(lamb, xi)) 103 | 104 | def residual_d(lamb,x,k,mu): 105 | ''' 106 | Calculates the residual of the moment approximation function. 107 | 108 | Parameters: 109 | lamb (array): an array of Lagrange constants used to approximate the distribution 110 | x (array): 111 | k (integer): order of the moment 112 | mu (array): an array of the known moments needed to approximate the distribution function 113 | 114 | Returns: 115 | rhs: the integrated right hand side of the moment approximation function 116 | ''' 117 | l_sum = [] 118 | for i in range(0,len(lamb)): 119 | l_sum.append( np.sum(integrand(x,lamb,i,discrete=True)) - mu[i] ) 120 | return np.array(l_sum) 121 | 122 | def maxent_reconstruct_d(rndvar, mu): 123 | ''' 124 | Computes the most likely distribution from the moments given using maximum entropy theorum. 125 | 126 | Parameters: 127 | rndvar (array): a list or array of known dependent variables. For example, for a 6-faced die, rndvar=[1,2,3,4,5,6] 128 | mu (array): vector of size m containing the known moments of a distribution. This does NOT assume that μ0 = 1. This vector contains moments μ_k starting with μ_0, μ_1, etc... For example, μ = [1,0,0] 129 | 130 | Returns: 131 | probabilites: vector of size b (from bnd[1]) containing the probabilities for the distribution 132 | lambsol: vector of lagrangian multipliers 133 | ''' 134 | lambguess = np.zeros(len(mu)) 135 | lambguess[0] = -np.log(np.sqrt(2*np.pi)) 136 | k = len(mu) 137 | lambsol = fsolve(residual_d, lambguess, args = (rndvar,k,mu)) 138 | probabilites = integrand(rndvar, lambsol, k=0, discrete=True) 139 | return probabilites, lambsol 140 | 141 | 142 | def residual_c(lamb, mu, bnds): 143 | ''' 144 | Calculates the residual of the moment approximation function. 145 | 146 | Parameters: 147 | lamb (array): an array of Lagrange constants used to approximate the distribution 148 | mu (array): an array of the known moments needed to approximate the distribution function 149 | bnds (tuple): support bounds 150 | 151 | Returns: 152 | rhs: the integrated right hand side of the moment approximation function 153 | ''' 154 | a = bnds[0] 155 | b = bnds[1] 156 | neqs = len(lamb) 157 | rhs = np.zeros(neqs) 158 | for k in range(0, neqs): 159 | rhs[k] = quad(integrand, a, b, args=(lamb, k))[0] - mu[k] 160 | return rhs 161 | 162 | def maxent_reconstruct_c(mu, bnds=[-np.inf, np.inf]): 163 | ''' 164 | Used to construct a continuous distribution from a limited number of known moments(μ). This function applies Maximum Entropy Theory in order to solve for the constraints found in the approximation equation that is given as an output. 165 | 166 | Parameters: 167 | μ: vector of size m containing the known moments of a distribution. This does NOT assume that μ0 = 1. This vector contains moments μ_k starting with μ_0, μ_1, etc... 168 | Ex. μ = [1,0,0] 169 | bnds: Support for the integration [a,b] 170 | ## It is important the bounds include roughly all non-zero values of the distribution that is being recreated ## 171 | 172 | Returns: 173 | Distribution Function: The recreated probability distribution function from the moment vector (μ) input given. requires a support to be ploted 174 | 175 | Example: 176 | >>> f, sol = maxent([1,0,0], [-1,1]) 177 | ''' 178 | neqs = len(mu) 179 | lambguess = np.zeros(neqs) # initialize guesses 180 | lambguess[0] = -np.log(np.sqrt(2*np.pi)) # set the first initial guess - this seems to work okay 181 | lambsol = fsolve(residual_c, lambguess, args=(mu,bnds), col_deriv=True) 182 | recon = lambda x: integrand(x, lambsol, k=0) 183 | return recon, lambsol 184 | 185 | def reconstruct(mu, rndvar=None, bnds=None): 186 | ''' 187 | This is the main function call to generate maximum entropy solutions. 188 | 189 | Parameters: 190 | mu (array): a list or array of known moments 191 | rndvar (array): optional - a list or array of known dependent variables. For example, for a 6-faced die, rndvar=[1,2,3,4,5,6]. If rndvar is provided, we will assume a discrete reconstruction. 192 | bnds (tuple): a tuple [a,b] containing the bounds or support of the reconstructed solution. This is only required for continuous distributions and will be neglected if rndvar is provided. 193 | 194 | Returns: 195 | recon: reconstructed distribution. If continuous, then `recon` is a Python function, `f(x)`. If discrete, then recon is an array of probabilities. 196 | lambsol (array): array containing the lagrangian multipliers 197 | 198 | Examples: 199 | ### reconstruct a discrete distribution 200 | >>> from pymaxent import * 201 | >>> mu = [1,3.5] 202 | >>> x = [1,2,3,4,5,6] 203 | >>> sol, lambdas = reconstruct(mu,rndvar=x) 204 | 205 | ### reconstruct a continuous distribution 206 | >>> from pymaxent import * 207 | >>> mu = [1,0,0.04] 208 | >>> sol, lambdas = reconstruct(mu,bnds=[-1,1]) 209 | >>> x = np.linspace(-1,1) 210 | >>> plot(x,sol(x)) 211 | ''' 212 | result = 0 213 | # Discrete case 214 | if rndvar is not None: 215 | rndvar = np.array(rndvar) # convert things to numpy arrays 216 | if bnds is not None: 217 | print('WARNING: You specified BOTH x and boundaries. I will assume this is a discrete distribution. If you want to calculate a continuous distribution, please specify bnd ONLY.') 218 | result = maxent_reconstruct_d(rndvar, mu) 219 | # Continuous case 220 | else: 221 | result = maxent_reconstruct_c(mu, bnds) 222 | return result 223 | --------------------------------------------------------------------------------