├── .gitignore ├── .pylintrc ├── .travis.yml ├── HISTORY.rst ├── LICENSE ├── README.rst ├── ace ├── __init__.py ├── ace.py ├── model.py ├── samples │ ├── __init__.py │ ├── breiman85.py │ ├── smoother_friedman82.py │ ├── supersmoother_friedman82.py │ └── wang04.py ├── smoother.py ├── supersmoother.py ├── tests │ ├── __init__.py │ ├── test_ace.py │ ├── test_model.py │ └── test_smoother.py └── validation │ ├── __init__.py │ └── validate_smoothers.py ├── doc ├── Makefile ├── _static │ ├── ace_input_wang04.png │ └── ace_transforms_wang04.png ├── conf.py ├── index.rst ├── make.bat ├── readme.rst └── samples.rst ├── profile_ace.py ├── requirements.txt ├── requirements_test.txt ├── setup.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | .project 2 | .pydevproject 3 | refs 4 | *.pyc 5 | ace/test/*.png 6 | ace/validation/*.png 7 | ace/samples/*.png 8 | *.f 9 | *.so 10 | *.o 11 | *.c 12 | *.doc 13 | *.sh 14 | *~ 15 | doc/_build 16 | doc/apidoc 17 | build 18 | dist 19 | ace.egg-info 20 | MANIFEST 21 | .settings* 22 | ace/tests/*.txt 23 | *.swp 24 | .tox 25 | -------------------------------------------------------------------------------- /.pylintrc: -------------------------------------------------------------------------------- 1 | [MASTER] 2 | # allow pylint to parse c-libraries 3 | extension-pkg-whitelist=numpy,scipy 4 | 5 | [FORMAT] 6 | max-line-length=120 7 | good-names=i,j,k,x,y,z,xi,yi,xt,xy,xj,yj,ri,N 8 | method-rgx=[a-z_][a-z0-9_]{2,50}$ 9 | function-rgx=[a-z_][a-z0-9_]{2,50}$ 10 | 11 | [MESSAGES CONTROL] 12 | disable= 13 | useless-object-inheritance, 14 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | sudo: false 2 | matrix: 3 | fast_finish: true 4 | include: 5 | - python: "2.7" 6 | env: TOXENV=py27 7 | - python: "3.4" 8 | env: TOXENV=py34 9 | - python: "3.6" 10 | env: TOXENV=lint 11 | - python: "3.5" 12 | env: TOXENV=py35 13 | - python: "3.6" 14 | env: TOXENV=py36 15 | 16 | cache: 17 | directories: 18 | - $HOME/.cache/pip 19 | install: pip install -U tox coveralls 20 | language: python 21 | script: tox 22 | after_success: coveralls 23 | -------------------------------------------------------------------------------- /HISTORY.rst: -------------------------------------------------------------------------------- 1 | History/Changelog 2 | ================= 3 | 4 | 0.3-3 5 | ----- 6 | - Fix invalid integer division (#16) 7 | 8 | 0.3-2 9 | ----- 10 | - Fixed divide-by-zero issue (#10) 11 | 12 | 0.3 13 | --- 14 | - Fixed too large of random seed in tests (#4) 15 | - Fixed README instructions (#8) 16 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2014 Nicholas W. Touran 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | =============== 2 | The ace Package 3 | =============== 4 | 5 | .. image:: https://travis-ci.org/partofthething/ace.svg?branch=develop 6 | :target: https://travis-ci.org/partofthething/ace 7 | 8 | ace is an implementation of the Alternating Conditional Expectation (ACE) algorithm [Breiman85]_, 9 | which can be used to find otherwise difficult-to-find relationships between predictors 10 | and responses and as a multivariate regression tool. 11 | 12 | The code for this project, as well as the issue tracker, etc. is 13 | `hosted on GitHub `_. 14 | The documentation is hosted at http://partofthething.com/ace. 15 | 16 | 17 | What is it? 18 | ----------- 19 | ACE can be used for a variety of purposes. With it, you can: 20 | 21 | - build easy-to-evaluate surrogate models of data. For example, if you are optimizing input 22 | parameters to a complex and long-running simulation, you can feed the results of a parameter 23 | sweep into ACE to get a model that will instantly give you predictions of results of any 24 | combination of input within the parameter range. 25 | 26 | - expose interesting and meaningful relations between predictors and responses from complicated 27 | data sets. For instance, if you have survey results from 1000 people and you and you want to 28 | see how one answer is related to a bunch of others, ACE will help you. 29 | 30 | The fascinating thing about ACE is that it is a *non-parametric* multivariate regression 31 | tool. This means that it doesn't make any assumptions about the functional form of the data. 32 | You may be used to fitting polynomials or lines to data. Well, ACE doesn't do that. It 33 | uses an iteration with a variable-span scatterplot smoother (implementing local least 34 | squares estimates) to figure out the structure of your data. As you'll see, that 35 | turns out to be a powerful difference. 36 | 37 | Installing it 38 | ------------- 39 | ace is available in the `Python Package Index `_, 40 | and can be installed simply with the following. 41 | 42 | On Linux:: 43 | 44 | sudo pip install ace 45 | 46 | On Windows, use:: 47 | 48 | pip install ace 49 | 50 | Directly from source:: 51 | 52 | git clone git@github.com:partofthething/ace.git 53 | cd ace 54 | python setup.py install 55 | 56 | .. note:: 57 | 58 | If you don't have git, you can just download the source directly from 59 | `here `_. 60 | 61 | You can verify that the installation completed successfully by running the automated test 62 | suite in the install directory:: 63 | 64 | python -m unittest discover -bv 65 | 66 | Using it 67 | -------- 68 | To use, get some sample data: 69 | 70 | .. code:: python 71 | 72 | from ace.samples import wang04 73 | x, y = wang04.build_sample_ace_problem_wang04(N=200) 74 | 75 | and run: 76 | 77 | .. code:: python 78 | 79 | from ace import model 80 | myace = model.Model() 81 | myace.build_model_from_xy(x, y) 82 | myace.eval([0.1, 0.2, 0.5, 0.3, 0.5]) 83 | 84 | For some plotting (matplotlib required), try: 85 | 86 | .. code:: python 87 | 88 | from ace import ace 89 | ace.plot_transforms(myace.ace, fname = 'mytransforms.pdf') 90 | myace.ace.write_transforms_to_file(fname = 'mytransforms.txt') 91 | 92 | Note that you could alternatively have loaded your data from a whitespace delimited 93 | text file: 94 | 95 | .. code:: python 96 | 97 | myace.build_model_from_txt(fname = 'myinput.txt') 98 | 99 | .. warning:: The more data points ACE is given as input, the better the results will be. 100 | Be careful with less than 50 data points or so. 101 | 102 | Demo 103 | ---- 104 | A combination of various functions with noise is shown below: 105 | 106 | .. have to use full path here to work in built docs and github-rendered README 107 | 108 | .. image:: https://partofthething.com/ace/_static/ace_input_wang04.png 109 | :alt: Plot of the input data, which is all over the place 110 | 111 | Given just those points and zero knowledge of the underlying functions, ACE comes back 112 | with this: 113 | 114 | .. image:: https://partofthething.com/ace/_static/ace_transforms_wang04.png 115 | :alt: Plot of the output transforms, which clearly show the underlying structure 116 | 117 | 118 | A longer version of this demo is available in the 119 | `Sample ACE Problems `_ section. 120 | 121 | Other details 122 | ------------- 123 | This implementation of ACE isn't as fast as the original FORTRAN version, but it can 124 | still crunch through a problem with 5 independent variables having 1000 observations each 125 | in on the order of 15 seconds. Not bad. 126 | 127 | ace also contains a pure-Python implementation of Friedman's SuperSmoother [Friedman82]_, 128 | the variable-span smoother mentioned above. This can be useful on its own 129 | for smoothing scatterplot data. 130 | 131 | History 132 | ------- 133 | The ACE algorithm was published in 1985 by Breiman and Friedman [Breiman85]_, and the original 134 | FORTRAN source code is available from `Friedman's webpage `_. 135 | 136 | Motivation 137 | ---------- 138 | Before this package, the ACE algorithm has only been available in Python by using the rpy2 module 139 | to load in the acepack package of the R statistical language. This package is a pure-Python 140 | re-write of the ACE algorithm based on the original publication, using modern software practices. 141 | This package is slower than the original FORTRAN code, but it is easier to understand. This package 142 | should be suitable for medium-weight data and as a learning tool. 143 | 144 | For the record, it is also quite easy to run the original FORTRAN code in Python using f2py. 145 | 146 | About the Author 147 | ---------------- 148 | This package was originated by Nick Touran, a nuclear engineer specializing in reactor physics. 149 | He was exposed to ACE by his thesis advisor, Professor John Lee, and used it in his 150 | Ph.D. dissertation to evaluate objective functions in a multidisciplinary 151 | design optimization study of nuclear reactor cores [Touran12]_. 152 | 153 | License 154 | ------- 155 | This package is released under the MIT License, `reproduced 156 | here `_. 157 | 158 | References 159 | ---------- 160 | .. [Breiman85] L. BREIMAN and J. H. FRIEDMAN, "Estimating optimal transformations for multiple regression and 161 | correlation," Journal of the American Statistical Association, 80, 580 (1985). 162 | `[Link1] `_ 163 | 164 | .. [Friedman82] J. H. FRIEDMAN and W. STUETZLE, "Smoothing of scatterplots," ORION-003, Stanford 165 | University, (1982). `[Link2] `_ 166 | 167 | .. [Wang04] D. WANG and M. MURPHY, "Estimating optimal transformations for multiple regression using the 168 | ACE algorithm," Journal of Data Science, 2, 329 (2004). 169 | `[Link3] `_ 170 | 171 | .. [Touran12] N. TOURAN, "A Modal Expansion Equilibrium Cycle Perturbation Method for 172 | Optimizing High Burnup Fast Reactors," Ph.D. dissertation, Univ. of Michigan, (2012). 173 | `[The Thesis] `_ 174 | 175 | 176 | -------------------------------------------------------------------------------- /ace/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | ace is a multivariate regression tool that solves alternating conditional expectations. 3 | 4 | See full documentation at http://partofthething.com/ace 5 | 6 | To use, get some sample data:: 7 | 8 | from ace.samples import wang04 9 | x, y = wang04.build_sample_ace_problem_wang04(N=200) 10 | 11 | and run:: 12 | 13 | from ace import model 14 | myace = model.Model() 15 | myace.build_model_from_xy(x, y) 16 | myace.eval([0.1, 0.2, 0.5, 0.3, 0.5]) 17 | 18 | For some plotting (matplotlib required), try:: 19 | 20 | from ace import ace 21 | ace.plot_transforms(myace, fname = 'mytransforms.pdf') 22 | myace.ace.write_transforms_to_file(fname = 'mytransforms.txt') 23 | 24 | """ 25 | 26 | import os.path 27 | from pkg_resources import get_distribution, DistributionNotFound 28 | 29 | def _get_version(): 30 | """ 31 | Groan single sourcing versions is a huge pain. 32 | 33 | For now we have to manually sync it between here and setup.py (for doc build) 34 | 35 | https://packaging.python.org/guides/single-sourcing-package-version/ 36 | """ 37 | try: 38 | _dist = get_distribution('ace') 39 | # Normalize case for Windows systems 40 | _dist_loc = os.path.normcase(_dist.location) # pylint: disable=no-member 41 | _here = os.path.normcase(__file__) 42 | if not _here.startswith(os.path.join(_dist_loc, 'ace')): 43 | # not installed, but there is another version that *is* 44 | raise DistributionNotFound 45 | except DistributionNotFound: 46 | version = '0.3.2' 47 | else: 48 | version = _dist.version # pylint: disable=no-member 49 | 50 | return version 51 | 52 | __version__ = _get_version() 53 | -------------------------------------------------------------------------------- /ace/ace.py: -------------------------------------------------------------------------------- 1 | r""" 2 | The Alternating Condtional Expectation (ACE) algorithm. 3 | 4 | ACE was invented by L. Breiman and J. Friedman [Breiman85]_. It is a powerful 5 | way to perform multidimensional regression without assuming 6 | any functional form of the model. Given a data set: 7 | 8 | :math:`y = f(X)` 9 | 10 | where :math:`X` is made up of a number of independent variables xi, ACE 11 | will tell you how :math:`y` varies vs. each of the individual independents :math:`xi`. 12 | This can be used to: 13 | 14 | * Understand the relative shape and magnitude of y's dependence on each xi 15 | * Produce a lightweight surrogate model of a more complex response 16 | * other stuff 17 | 18 | """ 19 | 20 | import numpy 21 | try: 22 | from matplotlib import pyplot as plt 23 | except ImportError: 24 | plt = None 25 | 26 | from .supersmoother import SuperSmoother 27 | from .smoother import perform_smooth 28 | 29 | 30 | MAX_OUTERS = 200 31 | 32 | 33 | class ACESolver(object): # pylint: disable=too-many-instance-attributes 34 | """The Alternating Conditional Expectation algorithm to perform regressions.""" 35 | 36 | def __init__(self): 37 | """Solver constructor.""" 38 | self._last_inner_error = float('inf') 39 | self._last_outer_error = float('inf') 40 | self.x = [] 41 | self.y = None 42 | self._xi_sorted = None 43 | self._yi_sorted = None 44 | self.x_transforms = None 45 | self.y_transform = None 46 | self._smoother_cls = SuperSmoother 47 | self._outer_iters = 0 48 | self._inner_iters = 0 49 | 50 | def specify_data_set(self, x_input, y_input): 51 | """ 52 | Define input to ACE. 53 | 54 | Parameters 55 | ---------- 56 | x_input : list 57 | list of iterables, one for each independent variable 58 | y_input : array 59 | the dependent observations 60 | 61 | """ 62 | self.x = x_input 63 | self.y = y_input 64 | 65 | def solve(self): 66 | """Run the ACE calculational loop.""" 67 | self._initialize() 68 | while self._outer_error_is_decreasing() and self._outer_iters < MAX_OUTERS: 69 | print('* Starting outer iteration {0:03d}. Current err = {1:12.5E}' 70 | ''.format(self._outer_iters, self._last_outer_error)) 71 | self._iterate_to_update_x_transforms() 72 | self._update_y_transform() 73 | self._outer_iters += 1 74 | 75 | def _initialize(self): 76 | """Set up and normalize initial data once input data is specified.""" 77 | self.y_transform = self.y - numpy.mean(self.y) 78 | self.y_transform /= numpy.std(self.y_transform) 79 | self.x_transforms = [numpy.zeros(len(self.y)) for _xi in self.x] 80 | self._compute_sorted_indices() 81 | 82 | def _compute_sorted_indices(self): 83 | """ 84 | Sort data from the perspective of each column. 85 | 86 | if self._x[0][3] is the 9th-smallest value in self._x[0], then _xi_sorted[3] = 8 87 | 88 | We only have to sort the data once. 89 | """ 90 | sorted_indices = [] 91 | for to_sort in [self.y] + self.x: 92 | data_w_indices = [(val, i) for (i, val) in enumerate(to_sort)] 93 | data_w_indices.sort() 94 | sorted_indices.append([i for val, i in data_w_indices]) 95 | # save in meaningful variable names 96 | self._yi_sorted = sorted_indices[0] # list (like self.y) 97 | self._xi_sorted = sorted_indices[1:] # list of lists (like self.x) 98 | 99 | def _outer_error_is_decreasing(self): 100 | """Return True if outer iteration error is decreasing.""" 101 | is_decreasing, self._last_outer_error = self._error_is_decreasing(self._last_outer_error) 102 | return is_decreasing 103 | 104 | def _error_is_decreasing(self, last_error): 105 | """Return True if current error is less than last_error.""" 106 | current_error = self._compute_error() 107 | is_decreasing = current_error < last_error 108 | return is_decreasing, current_error 109 | 110 | def _compute_error(self): 111 | """Compute unexplained error.""" 112 | sum_x = sum(self.x_transforms) 113 | err = sum((self.y_transform - sum_x) ** 2) / len(sum_x) 114 | return err 115 | 116 | def _iterate_to_update_x_transforms(self): 117 | """Perform the inner iteration.""" 118 | self._inner_iters = 0 119 | self._last_inner_error = float('inf') 120 | while self._inner_error_is_decreasing(): 121 | print(' Starting inner iteration {0:03d}. Current err = {1:12.5E}' 122 | ''.format(self._inner_iters, self._last_inner_error)) 123 | self._update_x_transforms() 124 | self._inner_iters += 1 125 | 126 | def _inner_error_is_decreasing(self): 127 | is_decreasing, self._last_inner_error = self._error_is_decreasing(self._last_inner_error) 128 | return is_decreasing 129 | 130 | def _update_x_transforms(self): 131 | """ 132 | Compute a new set of x-transform functions phik. 133 | 134 | phik(xk) = theta(y) - sum of phii(xi) over i!=k 135 | 136 | This is the first of the eponymous conditional expectations. The conditional 137 | expectations are computed using the SuperSmoother. 138 | """ 139 | # start by subtracting all transforms 140 | theta_minus_phis = self.y_transform - numpy.sum(self.x_transforms, axis=0) 141 | 142 | # add one transform at a time so as to exclude it from the subtracted sum 143 | for xtransform_index in range(len(self.x_transforms)): 144 | xtransform = self.x_transforms[xtransform_index] 145 | sorted_data_indices = self._xi_sorted[xtransform_index] 146 | xk_sorted = sort_vector(self.x[xtransform_index], sorted_data_indices) 147 | xtransform_sorted = sort_vector(xtransform, sorted_data_indices) 148 | theta_minus_phis_sorted = sort_vector(theta_minus_phis, sorted_data_indices) 149 | 150 | # minimize sums by just adding in the phik where i!=k here. 151 | to_smooth = theta_minus_phis_sorted + xtransform_sorted 152 | 153 | smoother = perform_smooth(xk_sorted, to_smooth, smoother_cls=self._smoother_cls) 154 | updated_x_transform_smooth = smoother.smooth_result 155 | updated_x_transform_smooth -= numpy.mean(updated_x_transform_smooth) 156 | 157 | # store updated transform in the order of the original data 158 | unsorted_xt = unsort_vector(updated_x_transform_smooth, sorted_data_indices) 159 | self.x_transforms[xtransform_index] = unsorted_xt 160 | 161 | # update main expession with new smooth. This was done in the original FORTRAN 162 | tmp_unsorted = unsort_vector(to_smooth, sorted_data_indices) 163 | theta_minus_phis = tmp_unsorted - unsorted_xt 164 | 165 | def _update_y_transform(self): 166 | """ 167 | Update the y-transform (theta). 168 | 169 | y-transform theta is forced to have mean = 0 and stddev = 1. 170 | 171 | This is the second conditional expectation 172 | """ 173 | # sort all phis wrt increasing y. 174 | sorted_data_indices = self._yi_sorted 175 | sorted_xtransforms = [] 176 | for xt in self.x_transforms: 177 | sorted_xt = sort_vector(xt, sorted_data_indices) 178 | sorted_xtransforms.append(sorted_xt) 179 | 180 | sum_of_x_transformations_choppy = numpy.sum(sorted_xtransforms, axis=0) 181 | y_sorted = sort_vector(self.y, sorted_data_indices) 182 | smooth = perform_smooth(y_sorted, sum_of_x_transformations_choppy, 183 | smoother_cls=self._smoother_cls) 184 | sum_of_x_transformations_smooth = smooth.smooth_result 185 | 186 | sum_of_x_transformations_smooth -= numpy.mean(sum_of_x_transformations_smooth) 187 | sum_of_x_transformations_smooth /= numpy.std(sum_of_x_transformations_smooth) 188 | 189 | # unsort to save in the original data 190 | self.y_transform = unsort_vector(sum_of_x_transformations_smooth, sorted_data_indices) 191 | 192 | def write_input_to_file(self, fname='ace_input.txt'): 193 | """Write y and x values used in this run to a space-delimited txt file.""" 194 | self._write_columns(fname, self.x, self.y) 195 | 196 | def write_transforms_to_file(self, fname='ace_transforms.txt'): 197 | """Write y and x transforms used in this run to a space-delimited txt file.""" 198 | self._write_columns(fname, self.x_transforms, self.y_transform) 199 | 200 | def _write_columns(self, fname, xvals, yvals): # pylint: disable=no-self-use 201 | with open(fname, 'w') as output_file: 202 | alldata = [yvals] + xvals 203 | for datai in zip(*alldata): 204 | yline = '{0: 15.9E} '.format(datai[0]) 205 | xline = ' '.join(['{0: 15.9E}'.format(xii) for xii in datai[1:]]) 206 | output_file.write(''.join([yline, xline, '\n'])) 207 | 208 | 209 | def sort_vector(data, indices_of_increasing): 210 | """Permutate 1-d data using given indices.""" 211 | return numpy.array([data[i] for i in indices_of_increasing]) 212 | 213 | 214 | def unsort_vector(data, indices_of_increasing): 215 | """Upermutate 1-D data that is sorted by indices_of_increasing.""" 216 | return numpy.array([data[indices_of_increasing.index(i)] for i in range(len(data))]) 217 | 218 | 219 | def plot_transforms(ace_model, fname='ace_transforms.png'): 220 | """Plot the transforms.""" 221 | if not plt: 222 | raise ImportError('Cannot plot without the matplotlib package') 223 | plt.rcParams.update({'font.size': 8}) 224 | plt.figure() 225 | num_cols = len(ace_model.x) // 2 + 1 226 | for i in range(len(ace_model.x)): 227 | plt.subplot(num_cols, 2, i + 1) 228 | plt.plot(ace_model.x[i], ace_model.x_transforms[i], '.', label='Phi {0}'.format(i)) 229 | plt.xlabel('x{0}'.format(i)) 230 | plt.ylabel('phi{0}'.format(i)) 231 | plt.subplot(num_cols, 2, i + 2) # pylint: disable=undefined-loop-variable 232 | plt.plot(ace_model.y, ace_model.y_transform, '.', label='Theta') 233 | plt.xlabel('y') 234 | plt.ylabel('theta') 235 | plt.tight_layout() 236 | 237 | if fname: 238 | plt.savefig(fname) 239 | return None 240 | return plt 241 | 242 | def plot_input(ace_model, fname='ace_input.png'): 243 | """Plot the transforms.""" 244 | if not plt: 245 | raise ImportError('Cannot plot without the matplotlib package') 246 | plt.rcParams.update({'font.size': 8}) 247 | plt.figure() 248 | num_cols = len(ace_model.x) // 2 + 1 249 | for i in range(len(ace_model.x)): 250 | plt.subplot(num_cols, 2, i + 1) 251 | plt.plot(ace_model.x[i], ace_model.y, '.') 252 | plt.xlabel('x{0}'.format(i)) 253 | plt.ylabel('y') 254 | 255 | plt.tight_layout() 256 | 257 | if fname: 258 | plt.savefig(fname) 259 | else: 260 | plt.show() 261 | -------------------------------------------------------------------------------- /ace/model.py: -------------------------------------------------------------------------------- 1 | """ 2 | The Model module is a front-end to the :py:mod:`ace.ace` module. 3 | 4 | ACE itself just gives transformations back as discontinuous data points. 5 | This module loads data, runs ACE, and then performs interpolations on the results, 6 | giving the user continuous functions that may be evaluated at any point 7 | within the range trained. 8 | 9 | This is a convenience/frontend/demo module. If you want to control ACE yourself, 10 | you may want to just use the ace module manually. 11 | 12 | """ 13 | 14 | from scipy.interpolate import interp1d 15 | 16 | from . import ace 17 | 18 | 19 | def read_column_data_from_txt(fname): 20 | """ 21 | Read data from a simple text file. 22 | 23 | Format should be just numbers. 24 | First column is the dependent variable. others are independent. 25 | Whitespace delimited. 26 | 27 | Returns 28 | ------- 29 | x_values : list 30 | List of x columns 31 | y_values : list 32 | list of y values 33 | 34 | """ 35 | datafile = open(fname) 36 | datarows = [] 37 | for line in datafile: 38 | datarows.append([float(li) for li in line.split()]) 39 | datacols = list(zip(*datarows)) 40 | x_values = datacols[1:] 41 | y_values = datacols[0] 42 | 43 | return x_values, y_values 44 | 45 | 46 | class Model(object): 47 | """A continuous model of data based on ACE regressions.""" 48 | 49 | def __init__(self): 50 | """Model constructor.""" 51 | self.ace = ace.ACESolver() 52 | self.phi_continuous = None 53 | self.inverse_theta_continuous = None 54 | 55 | def build_model_from_txt(self, fname): 56 | """ 57 | Construct the model and perform regressions based on data in a txt file. 58 | 59 | Parameters 60 | ---------- 61 | fname : str 62 | The name of the file to load. 63 | 64 | """ 65 | x_values, y_values = read_column_data_from_txt(fname) 66 | self.build_model_from_xy(x_values, y_values) 67 | 68 | def build_model_from_xy(self, x_values, y_values): 69 | """Construct the model and perform regressions based on x, y data.""" 70 | self.init_ace(x_values, y_values) 71 | self.run_ace() 72 | self.build_interpolators() 73 | 74 | def init_ace(self, x_values, y_values): 75 | """Specify data for the ACE solver object.""" 76 | self.ace.specify_data_set(x_values, y_values) 77 | 78 | def run_ace(self): 79 | """Perform the ACE calculation.""" 80 | self.ace.solve() 81 | 82 | def build_interpolators(self): 83 | """Compute 1-D interpolation functions for all the transforms so they're continuous..""" 84 | self.phi_continuous = [] 85 | for xi, phii in zip(self.ace.x, self.ace.x_transforms): 86 | self.phi_continuous.append( 87 | interp1d( 88 | x=xi, 89 | y=phii, 90 | bounds_error=False, 91 | fill_value=(min(phii), max(phii)))) 92 | self.inverse_theta_continuous = interp1d( 93 | x=self.ace.y_transform, 94 | y=self.ace.y, 95 | bounds_error=False, 96 | fill_value=(min(self.ace.y), max(self.ace.y))) 97 | 98 | def eval(self, x_values): 99 | """ 100 | Evaluate the ACE regression at any combination of independent variable values. 101 | 102 | Parameters 103 | ---------- 104 | x_values : iterable 105 | a float x-value for each independent variable, e.g. (1.5, 2.5) 106 | 107 | """ 108 | if len(x_values) != len(self.phi_continuous): 109 | raise ValueError( 110 | 'x_values must have length equal to the number of independent variables ' 111 | '({0}) rather than {1}.'.format( 112 | len(self.phi_continuous), len(x_values))) 113 | 114 | sum_phi = sum( 115 | [phi(xi) for phi, xi in zip(self.phi_continuous, x_values)]) 116 | return self.inverse_theta_continuous(sum_phi) 117 | 118 | 119 | if __name__ == '__main__': 120 | pass 121 | -------------------------------------------------------------------------------- /ace/samples/__init__.py: -------------------------------------------------------------------------------- 1 | """Sample ace and supersmoother problems from literature.""" 2 | -------------------------------------------------------------------------------- /ace/samples/breiman85.py: -------------------------------------------------------------------------------- 1 | """Run the Sample ACE problem from [Breiman85]_.""" 2 | 3 | import numpy.random 4 | import scipy.special 5 | 6 | from ace import ace 7 | 8 | 9 | def build_sample_ace_problem_breiman85(N=200): 10 | """Sample problem from Breiman 1985.""" 11 | x_cubed = numpy.random.standard_normal(N) 12 | x = scipy.special.cbrt(x_cubed) 13 | noise = numpy.random.standard_normal(N) 14 | y = numpy.exp((x ** 3.0) + noise) 15 | return [x], y 16 | 17 | 18 | def build_sample_ace_problem_breiman2(N=500): 19 | """Build sample problem y(x) = exp(sin(x)).""" 20 | x = numpy.linspace(0, 1, N) 21 | # x = numpy.random.uniform(0, 1, size=N) 22 | noise = numpy.random.standard_normal(N) 23 | y = numpy.exp(numpy.sin(2 * numpy.pi * x)) + 0.0 * noise 24 | return [x], y 25 | 26 | 27 | def run_breiman85(): 28 | """Run Breiman 85 sample.""" 29 | x, y = build_sample_ace_problem_breiman85(200) 30 | ace_solver = ace.ACESolver() 31 | ace_solver.specify_data_set(x, y) 32 | ace_solver.solve() 33 | try: 34 | ace.plot_transforms(ace_solver, 'sample_ace_breiman85.png') 35 | except ImportError: 36 | pass 37 | return ace_solver 38 | 39 | def run_breiman2(): 40 | """Run Breiman's other sample problem.""" 41 | x, y = build_sample_ace_problem_breiman2(500) 42 | ace_solver = ace.ACESolver() 43 | ace_solver.specify_data_set(x, y) 44 | ace_solver.solve() 45 | try: 46 | plt = ace.plot_transforms(ace_solver, None) 47 | except ImportError: 48 | pass 49 | 50 | plt.subplot(1, 2, 1) 51 | phi = numpy.sin(2.0 * numpy.pi * x[0]) 52 | plt.plot(x[0], phi, label='analytic') 53 | plt.legend() 54 | plt.subplot(1, 2, 2) 55 | y = numpy.exp(phi) 56 | plt.plot(y, phi, label='analytic') 57 | plt.legend(loc='lower right') 58 | # plt.show() 59 | plt.savefig('no_noise_linear_x.png') 60 | 61 | return ace_solver 62 | 63 | 64 | if __name__ == '__main__': 65 | run_breiman2() 66 | -------------------------------------------------------------------------------- /ace/samples/smoother_friedman82.py: -------------------------------------------------------------------------------- 1 | """Problem demonstrating supersmoother from [Friedman82]_.""" 2 | 3 | import math 4 | 5 | import numpy.random 6 | import matplotlib.pyplot as plt 7 | 8 | from ace import smoother 9 | 10 | 11 | def build_sample_smoother_problem_friedman82(N=200): 12 | """Sample problem from supersmoother publication.""" 13 | x = numpy.random.uniform(size=N) 14 | err = numpy.random.standard_normal(N) 15 | y = numpy.sin(2 * math.pi * (1 - x) ** 2) + x * err 16 | return x, y 17 | 18 | def run_friedman82_basic(): 19 | """Run Friedman's test of fixed-span smoothers from Figure 2b.""" 20 | x, y = build_sample_smoother_problem_friedman82() 21 | plt.figure() 22 | # plt.plot(x, y, '.', label='Data') 23 | for span in smoother.DEFAULT_SPANS: 24 | smooth = smoother.BasicFixedSpanSmoother() 25 | smooth.specify_data_set(x, y, sort_data=True) 26 | smooth.set_span(span) 27 | smooth.compute() 28 | plt.plot(x, smooth.smooth_result, '.', label='span = {0}'.format(span)) 29 | plt.legend(loc='upper left') 30 | plt.grid(color='0.7') 31 | plt.xlabel('x') 32 | plt.ylabel('y') 33 | plt.title('Demo of fixed-span smoothers from Friedman 82') 34 | plt.savefig('sample_friedman82.png') 35 | 36 | return smooth 37 | 38 | if __name__ == '__main__': 39 | run_friedman82_basic() 40 | -------------------------------------------------------------------------------- /ace/samples/supersmoother_friedman82.py: -------------------------------------------------------------------------------- 1 | """Problem demonstrating supersmoother from [Friedman82]_.""" 2 | 3 | import matplotlib.pyplot as plt 4 | 5 | from ace.samples import smoother_friedman82 6 | from ace.supersmoother import SuperSmoother 7 | 8 | def run_friedman82_super(): 9 | """Run Friedman's test of fixed-span smoothers from Figure 2b.""" 10 | x, y = smoother_friedman82.build_sample_smoother_problem_friedman82() 11 | plt.figure() 12 | smooth = SuperSmoother() 13 | smooth.specify_data_set(x, y, sort_data=True) 14 | smooth.compute() 15 | plt.plot(x, y, '.', label='Data') 16 | plt.plot(smooth.x, smooth.smooth_result, 'o', label='Smooth') 17 | plt.grid(color='0.7') 18 | plt.legend(loc='upper left') 19 | plt.title('Demo of SuperSmoother based on Friedman 82') 20 | plt.xlabel('x') 21 | plt.ylabel('y') 22 | plt.savefig('sample_supersmoother_friedman82.png') 23 | 24 | return smooth 25 | 26 | if __name__ == '__main__': 27 | run_friedman82_super() 28 | -------------------------------------------------------------------------------- /ace/samples/wang04.py: -------------------------------------------------------------------------------- 1 | """Run the Sample problem from [Wang04]_.""" 2 | 3 | import numpy 4 | 5 | from ace import ace 6 | 7 | def build_sample_ace_problem_wang04(N=100): 8 | """Build sample problem from Wang 2004.""" 9 | x = [numpy.random.uniform(-1, 1, size=N) 10 | for _i in range(0, 5)] 11 | noise = numpy.random.standard_normal(N) 12 | y = numpy.log(4.0 + numpy.sin(4 * x[0]) + numpy.abs(x[1]) + x[2] ** 2 + 13 | x[3] ** 3 + x[4] + 0.1 * noise) 14 | return x, y 15 | 16 | def run_wang04(): 17 | """Run sample problem.""" 18 | x, y = build_sample_ace_problem_wang04(N=200) 19 | ace_solver = ace.ACESolver() 20 | ace_solver.specify_data_set(x, y) 21 | ace_solver.solve() 22 | try: 23 | ace.plot_transforms(ace_solver, 'ace_transforms_wang04.png') 24 | ace.plot_input(ace_solver, 'ace_input_wang04.png') 25 | except ImportError: 26 | pass 27 | 28 | return ace_solver 29 | 30 | if __name__ == '__main__': 31 | run_wang04() 32 | -------------------------------------------------------------------------------- /ace/smoother.py: -------------------------------------------------------------------------------- 1 | """ 2 | Scatterplot smoother with a fixed span. 3 | 4 | Takes x,y scattered data and returns a set of 5 | (x,s) points that form a smoother curve fitting the data with moving least squares estimates. 6 | Similar to a moving average, but with better characteristics. The fundamental issue 7 | with this smoother is that the choice of span (window size) is not known in advance. 8 | The SuperSmoother uses these smoothers to figure out which span is optimal. 9 | 10 | This is a Python port of J. Friedman's 1982 fixed-span Smoother [Friedman82]_ 11 | 12 | Example:: 13 | 14 | s = Smoother() 15 | s.specify_data_set(x, y, sort_data = True) 16 | s.set_span(0.05) 17 | s.compute() 18 | smoothed_y = s.smooth_result 19 | 20 | """ 21 | 22 | import numpy 23 | import matplotlib.pyplot as plt 24 | 25 | TWEETER_SPAN = 0.05 26 | MID_SPAN = 0.2 27 | BASS_SPAN = 0.5 28 | 29 | DEFAULT_SPANS = (TWEETER_SPAN, MID_SPAN, BASS_SPAN) 30 | 31 | class Smoother(object): # pylint: disable=too-many-instance-attributes 32 | """Smoother that accepts data and produces smoother curves that fit the data.""" 33 | 34 | def __init__(self): 35 | """Smoother constructor.""" 36 | self.x = [] 37 | self.y = [] 38 | 39 | self._mean_x_in_window = 0.0 40 | self._mean_y_in_window = 0.0 41 | self._covariance_in_window = 0.0 42 | self._variance_in_window = 0.0 43 | 44 | self._span = None 45 | self.smooth_result = [] 46 | self.cross_validated_residual = None 47 | self.window_size = None 48 | self._original_index_of_xvalue = [] # for dealing w/ unsorted data 49 | self._window_bound_lower = 0 50 | self._x_in_window = [] 51 | self._y_in_window = [] 52 | self._neighbors_on_each_side = None 53 | 54 | def add_data_point_xy(self, x, y): 55 | """Add a new data point to the data set to be smoothed.""" 56 | self.x.append(x) 57 | self.y.append(y) 58 | 59 | def specify_data_set(self, x_input, y_input, sort_data=False): 60 | """ 61 | Fully define data by lists of x values and y values. 62 | 63 | This will sort them by increasing x but remember how to unsort them for providing results. 64 | 65 | Parameters 66 | ---------- 67 | x_input : iterable 68 | list of floats that represent x 69 | y_input : iterable 70 | list of floats that represent y(x) for each x 71 | sort_data : bool, optional 72 | If true, the data will be sorted by increasing x values. 73 | 74 | """ 75 | if sort_data: 76 | xy = sorted(zip(x_input, y_input)) 77 | x, y = zip(*xy) 78 | x_input_list = list(x_input) 79 | self._original_index_of_xvalue = [x_input_list.index(xi) for xi in x] 80 | if len(set(self._original_index_of_xvalue)) != len(x): 81 | raise RuntimeError('There are some non-unique x-values') 82 | else: 83 | x, y = x_input, y_input 84 | 85 | self.x = x 86 | self.y = y 87 | 88 | def set_span(self, span): 89 | """ 90 | Set the window-size for computing the least squares fit. 91 | 92 | Parameters 93 | ---------- 94 | span : float 95 | Fraction on data length N to be considered in smoothing 96 | 97 | """ 98 | self._span = span 99 | 100 | def compute(self): 101 | """Perform the smoothing operations.""" 102 | raise NotImplementedError 103 | 104 | def plot(self, fname=None): 105 | """ 106 | Plot the input data and resulting smooth. 107 | 108 | Parameters 109 | ---------- 110 | fname : str, optional 111 | name of file to produce. If none, will show interactively. 112 | 113 | """ 114 | plt.figure() 115 | xy = sorted(zip(self.x, self.smooth_result)) 116 | x, y = zip(*xy) 117 | plt.plot(x, y, '-') 118 | plt.plot(self.x, self.y, '.') 119 | if fname: 120 | plt.savefig(fname) 121 | else: 122 | plt.show() 123 | plt.close() 124 | 125 | def _store_unsorted_results(self, smooth, residual): 126 | """Convert sorted smooth/residual back to as-input order.""" 127 | if self._original_index_of_xvalue: 128 | # data was sorted. Unsort it here. 129 | self.smooth_result = numpy.zeros(len(self.y)) 130 | self.cross_validated_residual = numpy.zeros(len(residual)) 131 | original_x = numpy.zeros(len(self.y)) 132 | for i, (xval, smooth_val, residual_val) in enumerate(zip(self.x, smooth, residual)): 133 | original_index = self._original_index_of_xvalue[i] 134 | original_x[original_index] = xval 135 | self.smooth_result[original_index] = smooth_val 136 | self.cross_validated_residual[original_index] = residual_val 137 | self.x = original_x 138 | else: 139 | # no sorting was done. just apply results 140 | self.smooth_result = smooth 141 | self.cross_validated_residual = residual 142 | 143 | 144 | 145 | class BasicFixedSpanSmoother(Smoother): # pylint: disable=too-many-instance-attributes 146 | """ 147 | A basic fixed-span smoother. 148 | 149 | Simple least-squares linear local smoother. 150 | 151 | Uses fast updates of means, variances. 152 | """ 153 | 154 | def compute(self): 155 | """Perform the smoothing operations.""" 156 | self._compute_window_size() 157 | smooth = [] 158 | residual = [] 159 | 160 | x, y = self.x, self.y 161 | 162 | # step through x and y data with a window window_size wide. 163 | self._update_values_in_window() 164 | self._update_mean_in_window() 165 | self._update_variance_in_window() 166 | for i, (xi, yi) in enumerate(zip(x, y)): 167 | if ((i - self._neighbors_on_each_side) > 0.0 and 168 | (i + self._neighbors_on_each_side) < len(x)): 169 | self._advance_window() 170 | smooth_here = self._compute_smooth_during_construction(xi) 171 | residual_here = self._compute_cross_validated_residual_here(xi, yi, smooth_here) 172 | smooth.append(smooth_here) 173 | residual.append(residual_here) 174 | 175 | self._store_unsorted_results(smooth, residual) 176 | 177 | def _compute_window_size(self): 178 | """Determine characteristics of symmetric neighborhood with J/2 values on each side.""" 179 | self._neighbors_on_each_side = int(len(self.x) * self._span) // 2 180 | self.window_size = self._neighbors_on_each_side * 2 + 1 181 | if self.window_size <= 1: 182 | # cannot do averaging with 1 point in window. Force >=2 183 | self.window_size = 2 184 | 185 | def _update_values_in_window(self): 186 | """Update which values are in the current window.""" 187 | window_bound_upper = self._window_bound_lower + self.window_size 188 | self._x_in_window = self.x[self._window_bound_lower:window_bound_upper] 189 | self._y_in_window = self.y[self._window_bound_lower:window_bound_upper] 190 | 191 | def _update_mean_in_window(self): 192 | """ 193 | Compute mean in window the slow way. useful for first step. 194 | 195 | Considers all values in window 196 | 197 | See Also 198 | -------- 199 | _add_observation_to_means : fast update of mean for single observation addition 200 | _remove_observation_from_means : fast update of mean for single observation removal 201 | 202 | """ 203 | self._mean_x_in_window = numpy.mean(self._x_in_window) 204 | self._mean_y_in_window = numpy.mean(self._y_in_window) 205 | 206 | def _update_variance_in_window(self): 207 | """ 208 | Compute variance and covariance in window using all values in window (slow). 209 | 210 | See Also 211 | -------- 212 | _add_observation_to_variances : fast update for single observation addition 213 | _remove_observation_from_variances : fast update for single observation removal 214 | 215 | """ 216 | self._covariance_in_window = sum([(xj - self._mean_x_in_window) * 217 | (yj - self._mean_y_in_window) 218 | for xj, yj in zip(self._x_in_window, self._y_in_window)]) 219 | 220 | self._variance_in_window = sum([(xj - self._mean_x_in_window) ** 2 for xj 221 | in self._x_in_window]) 222 | 223 | def _advance_window(self): 224 | """Update values in current window and the current window means and variances.""" 225 | x_to_remove, y_to_remove = self._x_in_window[0], self._y_in_window[0] 226 | 227 | self._window_bound_lower += 1 228 | self._update_values_in_window() 229 | x_to_add, y_to_add = self._x_in_window[-1], self._y_in_window[-1] 230 | 231 | self._remove_observation(x_to_remove, y_to_remove) 232 | self._add_observation(x_to_add, y_to_add) 233 | 234 | def _remove_observation(self, x_to_remove, y_to_remove): 235 | """Remove observation from window, updating means/variance efficiently.""" 236 | self._remove_observation_from_variances(x_to_remove, y_to_remove) 237 | self._remove_observation_from_means(x_to_remove, y_to_remove) 238 | self.window_size -= 1 239 | 240 | def _add_observation(self, x_to_add, y_to_add): 241 | """Add observation to window, updating means/variance efficiently.""" 242 | self._add_observation_to_means(x_to_add, y_to_add) 243 | self._add_observation_to_variances(x_to_add, y_to_add) 244 | self.window_size += 1 245 | 246 | def _add_observation_to_means(self, xj, yj): 247 | """Update the means without recalculating for the addition of one observation.""" 248 | self._mean_x_in_window = ((self.window_size * self._mean_x_in_window + xj) / 249 | (self.window_size + 1.0)) 250 | self._mean_y_in_window = ((self.window_size * self._mean_y_in_window + yj) / 251 | (self.window_size + 1.0)) 252 | 253 | def _remove_observation_from_means(self, xj, yj): 254 | """Update the means without recalculating for the deletion of one observation.""" 255 | self._mean_x_in_window = ((self.window_size * self._mean_x_in_window - xj) / 256 | (self.window_size - 1.0)) 257 | self._mean_y_in_window = ((self.window_size * self._mean_y_in_window - yj) / 258 | (self.window_size - 1.0)) 259 | 260 | def _add_observation_to_variances(self, xj, yj): 261 | """ 262 | Quickly update the variance and co-variance for the addition of one observation. 263 | 264 | See Also 265 | -------- 266 | _update_variance_in_window : compute variance considering full window 267 | 268 | """ 269 | term1 = (self.window_size + 1.0) / self.window_size * (xj - self._mean_x_in_window) 270 | self._covariance_in_window += term1 * (yj - self._mean_y_in_window) 271 | self._variance_in_window += term1 * (xj - self._mean_x_in_window) 272 | 273 | def _remove_observation_from_variances(self, xj, yj): 274 | """Quickly update the variance and co-variance for the deletion of one observation.""" 275 | term1 = self.window_size / (self.window_size - 1.0) * (xj - self._mean_x_in_window) 276 | self._covariance_in_window -= term1 * (yj - self._mean_y_in_window) 277 | self._variance_in_window -= term1 * (xj - self._mean_x_in_window) 278 | 279 | def _compute_smooth_during_construction(self, xi): 280 | """ 281 | Evaluate value of smooth at x-value xi. 282 | 283 | Parameters 284 | ---------- 285 | xi : float 286 | Value of x where smooth value is desired 287 | 288 | Returns 289 | ------- 290 | smooth_here : float 291 | Value of smooth s(xi) 292 | 293 | """ 294 | if self._variance_in_window: 295 | beta = self._covariance_in_window / self._variance_in_window 296 | alpha = self._mean_y_in_window - beta * self._mean_x_in_window 297 | value_of_smooth_here = beta * (xi) + alpha 298 | else: 299 | value_of_smooth_here = 0.0 300 | return value_of_smooth_here 301 | 302 | def _compute_cross_validated_residual_here(self, xi, yi, smooth_here): 303 | """ 304 | Compute cross validated residual. 305 | 306 | This is the absolute residual from Eq. 9. in [1] 307 | """ 308 | denom = (1.0 - 1.0 / self.window_size - 309 | (xi - self._mean_x_in_window) ** 2 / 310 | self._variance_in_window) 311 | if denom == 0.0: 312 | # can happen with small data sets 313 | return 1.0 314 | return abs((yi - smooth_here) / denom) 315 | 316 | class BasicFixedSpanSmootherSlowUpdate(BasicFixedSpanSmoother): 317 | """Use slow means and variances at each step. Used to validate fast updates.""" 318 | 319 | def _advance_window(self): 320 | self._window_bound_lower += 1 321 | self._update_values_in_window() 322 | self._update_mean_in_window() 323 | self._update_variance_in_window() 324 | 325 | 326 | DEFAULT_BASIC_SMOOTHER = BasicFixedSpanSmoother # pylint: disable=invalid-name 327 | 328 | 329 | def perform_smooth(x_values, y_values, span=None, smoother_cls=None): 330 | """ 331 | Run the basic smoother (convenience function). 332 | 333 | Parameters 334 | ---------- 335 | x_values : iterable 336 | List of x value observations 337 | y_ values : iterable 338 | list of y value observations 339 | span : float, optional 340 | Fraction of data to use as the window 341 | smoother_cls : Class 342 | The class of smoother to use to smooth the data 343 | 344 | Returns 345 | ------- 346 | smoother : object 347 | The smoother object with results stored on it. 348 | 349 | """ 350 | if smoother_cls is None: 351 | smoother_cls = DEFAULT_BASIC_SMOOTHER 352 | smoother = smoother_cls() 353 | smoother.specify_data_set(x_values, y_values) 354 | smoother.set_span(span) 355 | smoother.compute() 356 | return smoother 357 | 358 | 359 | if __name__ == '__main__': 360 | pass 361 | -------------------------------------------------------------------------------- /ace/supersmoother.py: -------------------------------------------------------------------------------- 1 | """ 2 | A variable-span data smoother. 3 | 4 | This uses the fixed-span smoother to determine 5 | a changing optimal span for the data based on cross-validated residuals. It 6 | is an adaptive smoother that requires several passes over the data. 7 | 8 | The SuperSmoother provides a mechanism to evaluate the conditional expectations 9 | in the ACE algorithm. 10 | 11 | Based on [Friedman82]_. 12 | 13 | Example:: 14 | 15 | s = SuperSmoother() 16 | s.specify_data_set(x, y, sort_data = True) 17 | s.compute() 18 | smoothed_y = s.smooth_result 19 | """ 20 | 21 | import numpy 22 | from matplotlib import pyplot as plt 23 | 24 | from . import smoother 25 | from .smoother import DEFAULT_SPANS, MID_SPAN, BASS_SPAN, TWEETER_SPAN 26 | 27 | 28 | BASS_INDEX = DEFAULT_SPANS.index(BASS_SPAN) 29 | 30 | 31 | class SuperSmoother(smoother.Smoother): 32 | """Variable-span smoother.""" 33 | 34 | def __init__(self): 35 | """Construct a SuperSmoother.""" 36 | super(SuperSmoother, self).__init__() 37 | 38 | self._primary_smooths = [] 39 | self._residual_smooths = [] 40 | self._best_span_at_each_point = [] 41 | self._smoothed_best_spans = None 42 | self._bass_enhancement = 0.0 # should be between 0 and 10. 43 | 44 | def set_bass_enhancement(self, alpha): 45 | """ 46 | Bass enhancement amplifies the bass span. 47 | 48 | This gives the resulting smooth a smoother look, which is sometimes desirable if 49 | the underlying mechanisms are known to be smooth. 50 | """ 51 | self._bass_enhancement = alpha 52 | 53 | def compute(self): 54 | """Run the SuperSmoother.""" 55 | self._compute_primary_smooths() 56 | self._smooth_the_residuals() 57 | self._select_best_smooth_at_each_point() 58 | self._enhance_bass() 59 | self._smooth_best_span_estimates() 60 | self._apply_best_spans_to_primaries() 61 | self._smooth_interpolated_smooth() 62 | self._store_unsorted_results(self.smooth_result, numpy.zeros(len(self.smooth_result))) 63 | 64 | def _compute_primary_smooths(self): 65 | """Compute fixed-span smooths with all of the default spans.""" 66 | for span in DEFAULT_SPANS: 67 | smooth = smoother.perform_smooth(self.x, self.y, span) 68 | self._primary_smooths.append(smooth) 69 | 70 | def _smooth_the_residuals(self): 71 | """ 72 | Apply the MID_SPAN to the residuals of the primary smooths. 73 | 74 | "For stability reasons, it turns out to be a little better to smooth 75 | |r_{i}(J)| against xi" - [1] 76 | """ 77 | for primary_smooth in self._primary_smooths: 78 | smooth = smoother.perform_smooth(self.x, 79 | primary_smooth.cross_validated_residual, 80 | MID_SPAN) 81 | self._residual_smooths.append(smooth.smooth_result) 82 | 83 | def _select_best_smooth_at_each_point(self): 84 | """ 85 | Solve Eq (10) to find the best span for each observation. 86 | 87 | Stores index so we can easily grab the best residual smooth, primary smooth, etc. 88 | """ 89 | for residuals_i in zip(*self._residual_smooths): 90 | index_of_best_span = residuals_i.index(min(residuals_i)) 91 | self._best_span_at_each_point.append(DEFAULT_SPANS[index_of_best_span]) 92 | 93 | def _enhance_bass(self): 94 | """Update best span choices with bass enhancement as requested by user (Eq. 11).""" 95 | if not self._bass_enhancement: 96 | # like in supsmu, skip if alpha=0 97 | return 98 | bass_span = DEFAULT_SPANS[BASS_INDEX] 99 | enhanced_spans = [] 100 | for i, best_span_here in enumerate(self._best_span_at_each_point): 101 | best_smooth_index = DEFAULT_SPANS.index(best_span_here) 102 | best_span = DEFAULT_SPANS[best_smooth_index] 103 | best_span_residual = self._residual_smooths[best_smooth_index][i] 104 | bass_span_residual = self._residual_smooths[BASS_INDEX][i] 105 | if 0 < best_span_residual < bass_span_residual: 106 | ri = best_span_residual / bass_span_residual 107 | bass_factor = ri ** (10.0 - self._bass_enhancement) 108 | enhanced_spans.append(best_span + (bass_span - best_span) * bass_factor) 109 | else: 110 | enhanced_spans.append(best_span) 111 | self._best_span_at_each_point = enhanced_spans 112 | 113 | def _smooth_best_span_estimates(self): 114 | """Apply a MID_SPAN smooth to the best span estimates at each observation.""" 115 | self._smoothed_best_spans = smoother.perform_smooth(self.x, 116 | self._best_span_at_each_point, 117 | MID_SPAN) 118 | 119 | def _apply_best_spans_to_primaries(self): 120 | """ 121 | Apply best spans. 122 | 123 | Given the best span, interpolate to compute the best smoothed value 124 | at each observation. 125 | """ 126 | self.smooth_result = [] 127 | for xi, best_span in enumerate(self._smoothed_best_spans.smooth_result): 128 | primary_values = [s.smooth_result[xi] for s in self._primary_smooths] 129 | # pylint: disable=no-member 130 | best_value = numpy.interp(best_span, DEFAULT_SPANS, primary_values) 131 | self.smooth_result.append(best_value) 132 | 133 | def _smooth_interpolated_smooth(self): 134 | """ 135 | Smooth interpolated results with tweeter span. 136 | 137 | A final step of the supersmoother is to smooth the interpolated values with 138 | the tweeter span. This is done in Breiman's supsmu.f but is not explicitly 139 | discussed in the publication. This step is necessary to match 140 | the FORTRAN version perfectly. 141 | """ 142 | smoothed_results = smoother.perform_smooth(self.x, 143 | self.smooth_result, 144 | TWEETER_SPAN) 145 | self.smooth_result = smoothed_results.smooth_result 146 | 147 | class SuperSmootherWithPlots(SuperSmoother): 148 | """Auxiliary subclass for researching/understanding the SuperSmoother.""" 149 | 150 | def _compute_primary_smooths(self): 151 | super(SuperSmootherWithPlots, self)._compute_primary_smooths() 152 | plt.figure() 153 | for smooth in self._primary_smooths: 154 | plt.plot(self.x, smooth.smooth_result) 155 | plt.plot(self.x, self.y, '.') 156 | plt.savefig('primary_smooths.png') 157 | plt.close() 158 | 159 | def _smooth_the_residuals(self): 160 | super(SuperSmootherWithPlots, self)._smooth_the_residuals() 161 | plt.figure() 162 | for residual, span in zip(self._residual_smooths, smoother.DEFAULT_SPANS): 163 | plt.plot(self.x, residual, label='{0}'.format(span)) 164 | plt.legend(loc='upper left') 165 | plt.savefig('residual_smooths.png') 166 | plt.close() 167 | 168 | def _select_best_smooth_at_each_point(self): 169 | super(SuperSmootherWithPlots, self)._select_best_smooth_at_each_point() 170 | plt.figure() 171 | plt.plot(self.x, self._best_span_at_each_point, label='Fresh') 172 | 173 | def _enhance_bass(self): 174 | super(SuperSmootherWithPlots, self)._enhance_bass() 175 | plt.plot(self.x, self._best_span_at_each_point, label='Enhanced bass') 176 | 177 | def _smooth_best_span_estimates(self): 178 | super(SuperSmootherWithPlots, self)._smooth_best_span_estimates() 179 | plt.plot(self.x, self._smoothed_best_spans.smooth_result, label='Smoothed') 180 | plt.legend(loc='upper left') 181 | plt.savefig('best_spans.png') 182 | plt.close() 183 | -------------------------------------------------------------------------------- /ace/tests/__init__.py: -------------------------------------------------------------------------------- 1 | """Unit test suite for ace.""" 2 | -------------------------------------------------------------------------------- /ace/tests/test_ace.py: -------------------------------------------------------------------------------- 1 | """ 2 | Unit tests for ACE methods. 3 | 4 | These implicitly cover the SuperSmoother as well, but they don't validate it. 5 | """ 6 | 7 | import unittest 8 | 9 | import ace.ace 10 | import ace.samples.breiman85 11 | 12 | # pylint: disable=protected-access, missing-docstring 13 | 14 | class TestAce(unittest.TestCase): 15 | """Tests.""" 16 | 17 | def setUp(self): 18 | self.ace = ace.ace.ACESolver() 19 | x, y = ace.samples.breiman85.build_sample_ace_problem_breiman85() 20 | self.ace.specify_data_set(x, y) 21 | self.ace._initialize() 22 | 23 | def test_compute_sorted_indices(self): 24 | yprevious = self.ace.y[self.ace._yi_sorted[0]] 25 | for yi in self.ace._yi_sorted[1:]: 26 | yhere = self.ace.y[yi] 27 | self.assertGreater(yhere, yprevious) 28 | yprevious = yhere 29 | xprevious = self.ace.x[0][self.ace._xi_sorted[0][0]] 30 | for xi in self.ace._xi_sorted[1:]: 31 | xhere = self.ace.x[xi] 32 | self.assertGreater(xhere, xprevious) 33 | xprevious = xhere 34 | 35 | def test_error_is_decreasing(self): 36 | err = self.ace._compute_error() 37 | self.assertFalse(self.ace._error_is_decreasing(err)[0]) 38 | 39 | def test_compute_error(self): 40 | err = self.ace._compute_error() 41 | self.assertNotAlmostEqual(err, 0.0) 42 | 43 | def test_update_x_transforms(self): 44 | err = self.ace._compute_error() 45 | self.ace._update_x_transforms() 46 | self.assertTrue(self.ace._error_is_decreasing(err)[0]) 47 | 48 | def test_update_y_transform(self): 49 | err = self.ace._compute_error() 50 | self.ace._update_x_transforms() 51 | self.ace._update_y_transform() 52 | self.assertTrue(self.ace._error_is_decreasing(err)[0]) 53 | 54 | def test_sort_vector(self): 55 | data = [5, 1, 4, 6] 56 | increasing = [1, 2, 0, 3] 57 | dsort = ace.ace.sort_vector(data, increasing) 58 | for item1, item2 in zip(sorted(data), dsort): 59 | self.assertEqual(item1, item2) 60 | 61 | def test_unsort_vector(self): 62 | unsorted = [5, 1, 4, 6] 63 | data = [1, 4, 5, 6] 64 | increasing = [1, 2, 0, 3] 65 | dunsort = ace.ace.unsort_vector(data, increasing) 66 | for item1, item2 in zip(dunsort, unsorted): 67 | self.assertEqual(item1, item2) 68 | 69 | 70 | if __name__ == "__main__": 71 | # import sys;sys.argv = ['', 'Test.testName'] 72 | unittest.main() 73 | -------------------------------------------------------------------------------- /ace/tests/test_model.py: -------------------------------------------------------------------------------- 1 | """Unit tests for ace model.""" 2 | 3 | import unittest 4 | import os 5 | 6 | from ace import model 7 | from ace.samples import breiman85 8 | from ace.samples import wang04 9 | 10 | # pylint: disable=protected-access, missing-docstring 11 | 12 | class TestModel(unittest.TestCase): 13 | 14 | def setUp(self): 15 | self.model = model.Model() 16 | 17 | def tearDown(self): 18 | pass 19 | 20 | def test_build_model_from_xy(self): 21 | x, y = breiman85.build_sample_ace_problem_breiman85() 22 | self.model.build_model_from_xy(x, y) 23 | 24 | def test_eval_1d(self): 25 | x, y = breiman85.build_sample_ace_problem_breiman85() 26 | self.model.build_model_from_xy(x, y) 27 | val = self.model.eval([0.5]) 28 | self.assertGreater(val, 0.0) 29 | 30 | def test_eval_multiple(self): 31 | x, y = wang04.build_sample_ace_problem_wang04() 32 | self.model.build_model_from_xy(x, y) 33 | val = self.model.eval([0.5, 0.3, 0.2, 0.1, 0.0]) 34 | self.assertGreater(val, 0.0) 35 | 36 | def test_read_column_data_from_txt(self): 37 | x, y = breiman85.build_sample_ace_problem_breiman85() 38 | self.model.build_model_from_xy(x, y) 39 | fname = os.path.join(os.path.dirname(__file__), 'sample_xy_input.txt') 40 | self.model.ace.write_input_to_file(fname) 41 | 42 | model2 = model.Model() 43 | model2.build_model_from_txt(fname) 44 | 45 | val = self.model.eval([0.5]) 46 | val2 = model2.eval([0.5]) 47 | self.assertAlmostEqual(val, val2, 2) 48 | 49 | model2.ace.write_transforms_to_file() 50 | 51 | def test_smaller_dataset(self): 52 | x, y = wang04.build_sample_ace_problem_wang04(N=50) 53 | self.model.build_model_from_xy(x, y) 54 | val = self.model.eval([0.5, 0.3, 0.2, 0.1, 0.0]) 55 | self.assertGreater(val, 0.0) 56 | 57 | 58 | if __name__ == "__main__": 59 | unittest.main() 60 | -------------------------------------------------------------------------------- /ace/tests/test_smoother.py: -------------------------------------------------------------------------------- 1 | """Smoother unit tests""" 2 | 3 | import unittest 4 | 5 | from ace import smoother 6 | 7 | # pylint: disable=protected-access, missing-docstring 8 | 9 | class TestSmoother(unittest.TestCase): 10 | def setUp(self): 11 | self.smoother = smoother.BasicFixedSpanSmoother() 12 | self.x_data = [1.0, 2.0, 3.0, 4.0] 13 | self.y_data = [4.0, 5.0, 6.0, 7.0] 14 | self.smoother.specify_data_set(self.x_data, self.y_data) 15 | self.smoother.window_size = 3 16 | self.smoother._update_values_in_window() 17 | self.smoother._update_mean_in_window() 18 | self.smoother._update_variance_in_window() 19 | 20 | 21 | def test_mean(self): 22 | size = self.smoother.window_size 23 | self.assertAlmostEqual(self.smoother._mean_x_in_window, 24 | sum(self.x_data[:size]) / len(self.x_data[:size])) 25 | 26 | self.assertAlmostEqual(self.smoother._mean_y_in_window, 27 | sum(self.y_data[:size]) / len(self.y_data[:size])) 28 | 29 | def test_mean_on_addition_of_observation(self): 30 | """Make sure things work when we add an observation.""" 31 | self.smoother._add_observation_to_means(7, 8) 32 | size = self.smoother.window_size 33 | self.assertAlmostEqual(self.smoother._mean_x_in_window, 34 | (sum(self.x_data[:size]) + 7.0) / 35 | (self.smoother.window_size + 1.0)) 36 | 37 | self.assertAlmostEqual(self.smoother._mean_y_in_window, 38 | (sum(self.y_data[:size]) + 8.0) / 39 | (self.smoother.window_size + 1.0)) 40 | 41 | def test_mean_on_removal_of_observation(self): 42 | """Make sure things work when we remove an observation.""" 43 | self.smoother._remove_observation_from_means(3, 6) 44 | 45 | self.assertAlmostEqual(self.smoother._mean_x_in_window, 46 | sum(self.x_data[:2]) / 47 | (self.smoother.window_size - 1.0)) 48 | 49 | self.assertAlmostEqual(self.smoother._mean_y_in_window, 50 | (sum(self.y_data[:2])) / 51 | (self.smoother.window_size - 1.0)) 52 | 53 | def test_variance_on_removal_of_observation(self): 54 | """Make sure variance and covariance work when we remove an observation quickly.""" 55 | self.smoother._remove_observation(3, 6) 56 | 57 | cov_from_update = self.smoother._covariance_in_window 58 | var_from_update = self.smoother._variance_in_window 59 | 60 | self.smoother._update_values_in_window() 61 | self.smoother._update_mean_in_window() 62 | self.smoother._update_variance_in_window() 63 | 64 | self.assertAlmostEqual(cov_from_update, self.smoother._covariance_in_window) 65 | self.assertAlmostEqual(var_from_update, self.smoother._variance_in_window) 66 | 67 | def test_variance_on_addition_of_observation(self): 68 | """Make sure variance and covariance work when we remove an observation quickly.""" 69 | self.smoother._add_observation(4, 7) 70 | cov_from_update = self.smoother._covariance_in_window 71 | var_from_update = self.smoother._variance_in_window 72 | 73 | self.smoother._update_values_in_window() 74 | self.smoother._update_mean_in_window() 75 | self.smoother._update_variance_in_window() 76 | 77 | self.assertAlmostEqual(cov_from_update, self.smoother._covariance_in_window) 78 | self.assertAlmostEqual(var_from_update, self.smoother._variance_in_window) 79 | 80 | def test_advance_window(self): 81 | self.assertIn(1.0, self.smoother._x_in_window) 82 | lowerbound = self.smoother._window_bound_lower 83 | self.smoother._advance_window() 84 | self.assertEqual(lowerbound + 1, self.smoother._window_bound_lower) 85 | self.assertNotIn(1.0, self.smoother._x_in_window) 86 | self.assertIn(4.0, self.smoother._x_in_window) 87 | 88 | def test_compute_smooth_during_construction(self): 89 | # test data is linear, so we just make sure we're on the line 90 | val = self.smoother._compute_smooth_during_construction(2.5) 91 | self.assertAlmostEqual(val, 5.5) 92 | 93 | def test_compute_cross_validated_residual_here(self): 94 | # weak test. Hard to do analytically without just repeating the method 95 | residual = self.smoother._compute_cross_validated_residual_here(2.5, 5.6, 5.5) 96 | self.assertNotEqual(residual, 0.0) 97 | 98 | if __name__ == "__main__": 99 | # import sys;sys.argv = ['', 'Test.testName'] 100 | unittest.main() 101 | -------------------------------------------------------------------------------- /ace/validation/__init__.py: -------------------------------------------------------------------------------- 1 | """Validation code.""" 2 | -------------------------------------------------------------------------------- /ace/validation/validate_smoothers.py: -------------------------------------------------------------------------------- 1 | """ 2 | A few validation problems to make sure the smoothers are working as expected. 3 | 4 | These depend on the supsmu module, which was created using f2py from Breiman's supsmu.f 5 | """ 6 | 7 | import matplotlib.pyplot as plt 8 | import numpy 9 | 10 | from ace.samples import smoother_friedman82 11 | import ace.smoother as smoother 12 | import ace.supersmoother as supersmoother 13 | 14 | # pylint: disable=protected-access, missing-docstring 15 | 16 | try: 17 | import mace 18 | except ImportError: 19 | print("WARNING: An F2Pyd version of Breiman's supsmu is not available. " 20 | "Validations will not work") 21 | raise 22 | 23 | def validate_basic_smoother(): 24 | """Run Friedman's test from Figure 2b.""" 25 | x, y = sort_data(*smoother_friedman82.build_sample_smoother_problem_friedman82()) 26 | plt.figure() 27 | # plt.plot(x, y, '.', label='Data') 28 | for span in smoother.DEFAULT_SPANS: 29 | my_smoother = smoother.perform_smooth(x, y, span) 30 | friedman_smooth, _resids = run_friedman_smooth(x, y, span) 31 | plt.plot(x, my_smoother.smooth_result, '.-', label='pyace span = {0}'.format(span)) 32 | plt.plot(x, friedman_smooth, '.-', label='Friedman span = {0}'.format(span)) 33 | finish_plot() 34 | 35 | 36 | def validate_basic_smoother_resid(): 37 | """Compare residuals.""" 38 | x, y = sort_data(*smoother_friedman82.build_sample_smoother_problem_friedman82()) 39 | plt.figure() 40 | for span in smoother.DEFAULT_SPANS: 41 | my_smoother = smoother.perform_smooth(x, y, span) 42 | _friedman_smooth, resids = run_friedman_smooth(x, y, span) # pylint: disable=unused-variable 43 | plt.plot(x, my_smoother.cross_validated_residual, '.-', 44 | label='pyace span = {0}'.format(span)) 45 | plt.plot(x, resids, '.-', label='Friedman span = {0}'.format(span)) 46 | finish_plot() 47 | 48 | def validate_supersmoother(): 49 | """Validate the supersmoother.""" 50 | x, y = smoother_friedman82.build_sample_smoother_problem_friedman82() 51 | x, y = sort_data(x, y) 52 | my_smoother = smoother.perform_smooth(x, y, smoother_cls=supersmoother.SuperSmootherWithPlots) 53 | # smoother.DEFAULT_BASIC_SMOOTHER = BasicFixedSpanSmootherBreiman 54 | supsmu_result = run_freidman_supsmu(x, y, bass_enhancement=0.0) 55 | mace_result = run_mace_smothr(x, y, bass_enhancement=0.0) 56 | plt.plot(x, y, '.', label='Data') 57 | plt.plot(x, my_smoother.smooth_result, '-', label='pyace') 58 | plt.plot(x, supsmu_result, '--', label='SUPSMU') 59 | plt.plot(x, mace_result, ':', label='SMOOTH') 60 | plt.legend() 61 | plt.savefig('supersmoother_validation.png') 62 | 63 | def validate_supersmoother_bass(): 64 | """Validate the supersmoother with extra bass.""" 65 | x, y = smoother_friedman82.build_sample_smoother_problem_friedman82() 66 | plt.figure() 67 | plt.plot(x, y, '.', label='Data') 68 | for bass in range(0, 10, 3): 69 | smooth = supersmoother.SuperSmoother() 70 | smooth.set_bass_enhancement(bass) 71 | smooth.specify_data_set(x, y) 72 | smooth.compute() 73 | plt.plot(x, smooth.smooth_result, '.', label='Bass = {0}'.format(bass)) 74 | # pylab.plot(self.x, smoother.smooth_result, label='Bass = {0}'.format(bass)) 75 | finish_plot() 76 | 77 | def validate_average_best_span(): 78 | """Figure 2d? from Friedman.""" 79 | N = 200 80 | num_trials = 400 81 | avg = numpy.zeros(N) 82 | for i in range(num_trials): 83 | x, y = smoother_friedman82.build_sample_smoother_problem_friedman82(N=N) 84 | my_smoother = smoother.perform_smooth( 85 | x, y, smoother_cls=supersmoother.SuperSmoother 86 | ) 87 | avg += my_smoother._smoothed_best_spans.smooth_result 88 | if not (i + 1) % 20: 89 | print(i + 1) 90 | avg /= num_trials 91 | plt.plot(my_smoother.x, avg, '.', label='Average JCV') 92 | finish_plot() 93 | 94 | def validate_known_curve(): 95 | """Validate on a sin function.""" 96 | plt.figure() 97 | N = 100 98 | x = numpy.linspace(-1, 1, N) 99 | y = numpy.sin(4 * x) 100 | smoother.DEFAULT_BASIC_SMOOTHER = smoother.BasicFixedSpanSmootherSlowUpdate 101 | smooth = smoother.perform_smooth(x, y, smoother_cls=supersmoother.SuperSmoother) 102 | plt.plot(x, smooth.smooth_result, label='Slow') 103 | smoother.DEFAULT_BASIC_SMOOTHER = smoother.BasicFixedSpanSmoother 104 | smooth = smoother.perform_smooth(x, y, smoother_cls=supersmoother.SuperSmoother) 105 | plt.plot(x, smooth.smooth_result, label='Fast') 106 | plt.plot(x, y, '.', label='data') 107 | plt.legend() 108 | plt.show() 109 | 110 | def finish_plot(): 111 | """Help with plotting.""" 112 | plt.legend() 113 | plt.grid(color='0.7') 114 | plt.xlabel('x') 115 | plt.ylabel('y') 116 | plt.show() 117 | 118 | def run_freidman_supsmu(x, y, bass_enhancement=0.0): 119 | """Run the FORTRAN supersmoother.""" 120 | N = len(x) 121 | weight = numpy.ones(N) 122 | results = numpy.zeros(N) 123 | flags = numpy.zeros((N, 7)) 124 | mace.supsmu(x, y, weight, 1, 0.0, bass_enhancement, results, flags) 125 | return results 126 | 127 | def run_friedman_smooth(x, y, span): 128 | """Run the FORTRAN smoother.""" 129 | N = len(x) 130 | weight = numpy.ones(N) 131 | results = numpy.zeros(N) 132 | residuals = numpy.zeros(N) 133 | mace.smooth(x, y, weight, span, 1, 1e-7, results, residuals) 134 | return results, residuals 135 | 136 | def run_mace_smothr(x, y, bass_enhancement=0.0): # pylint: disable=unused-argument 137 | """Run the FORTRAN SMOTHR.""" 138 | N = len(x) 139 | weight = numpy.ones(N) 140 | results = numpy.zeros(N) 141 | flags = numpy.zeros((N, 7)) 142 | mace.smothr(1, x, y, weight, results, flags) 143 | return results 144 | 145 | class BasicFixedSpanSmootherBreiman(smoother.Smoother): 146 | """Runs FORTRAN Smooth.""" 147 | 148 | def compute(self): 149 | """Run smoother.""" 150 | self.smooth_result, self.cross_validated_residual = run_friedman_smooth( 151 | self.x, self.y, self._span 152 | ) 153 | 154 | class SuperSmootherBreiman(smoother.Smoother): 155 | """Run FORTRAN Supersmoother.""" 156 | 157 | def compute(self): 158 | """Run SuperSmoother.""" 159 | self.smooth_result = run_freidman_supsmu(self.x, self.y) 160 | self._store_unsorted_results(self.smooth_result, numpy.zeros(len(self.smooth_result))) 161 | 162 | def sort_data(x, y): 163 | """Sort the data.""" 164 | xy = sorted(zip(x, y)) 165 | x, y = zip(*xy) 166 | return x, y 167 | 168 | if __name__ == '__main__': 169 | validate_basic_smoother() 170 | # validate_basic_smoother_resid() 171 | #validate_supersmoother() 172 | -------------------------------------------------------------------------------- /doc/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # User-friendly check for sphinx-build 11 | ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) 12 | $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) 13 | endif 14 | 15 | # Internal variables. 16 | PAPEROPT_a4 = -D latex_paper_size=a4 17 | PAPEROPT_letter = -D latex_paper_size=letter 18 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 19 | # the i18n builder cannot share the environment and doctrees with the others 20 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 21 | 22 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 23 | 24 | help: 25 | @echo "Please use \`make ' where is one of" 26 | @echo " html to make standalone HTML files" 27 | @echo " dirhtml to make HTML files named index.html in directories" 28 | @echo " singlehtml to make a single large HTML file" 29 | @echo " pickle to make pickle files" 30 | @echo " json to make JSON files" 31 | @echo " htmlhelp to make HTML files and a HTML help project" 32 | @echo " qthelp to make HTML files and a qthelp project" 33 | @echo " devhelp to make HTML files and a Devhelp project" 34 | @echo " epub to make an epub" 35 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 36 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 37 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 38 | @echo " text to make text files" 39 | @echo " man to make manual pages" 40 | @echo " texinfo to make Texinfo files" 41 | @echo " info to make Texinfo files and run them through makeinfo" 42 | @echo " gettext to make PO message catalogs" 43 | @echo " changes to make an overview of all changed/added/deprecated items" 44 | @echo " xml to make Docutils-native XML files" 45 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 46 | @echo " linkcheck to check all external links for integrity" 47 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 48 | 49 | clean: 50 | rm -rf $(BUILDDIR)/* 51 | 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | dirhtml: 58 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 59 | @echo 60 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 61 | 62 | singlehtml: 63 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 64 | @echo 65 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 66 | 67 | pickle: 68 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 69 | @echo 70 | @echo "Build finished; now you can process the pickle files." 71 | 72 | json: 73 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 74 | @echo 75 | @echo "Build finished; now you can process the JSON files." 76 | 77 | htmlhelp: 78 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 79 | @echo 80 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 81 | ".hhp project file in $(BUILDDIR)/htmlhelp." 82 | 83 | qthelp: 84 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 85 | @echo 86 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 87 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 88 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/ace.qhcp" 89 | @echo "To view the help file:" 90 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/ace.qhc" 91 | 92 | devhelp: 93 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 94 | @echo 95 | @echo "Build finished." 96 | @echo "To view the help file:" 97 | @echo "# mkdir -p $$HOME/.local/share/devhelp/ace" 98 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/ace" 99 | @echo "# devhelp" 100 | 101 | epub: 102 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 103 | @echo 104 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 105 | 106 | latex: 107 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 108 | @echo 109 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 110 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 111 | "(use \`make latexpdf' here to do that automatically)." 112 | 113 | latexpdf: 114 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 115 | @echo "Running LaTeX files through pdflatex..." 116 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 117 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 118 | 119 | latexpdfja: 120 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 121 | @echo "Running LaTeX files through platex and dvipdfmx..." 122 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 123 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 124 | 125 | text: 126 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 127 | @echo 128 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 129 | 130 | man: 131 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 132 | @echo 133 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 134 | 135 | texinfo: 136 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 137 | @echo 138 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 139 | @echo "Run \`make' in that directory to run these through makeinfo" \ 140 | "(use \`make info' here to do that automatically)." 141 | 142 | info: 143 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 144 | @echo "Running Texinfo files through makeinfo..." 145 | make -C $(BUILDDIR)/texinfo info 146 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 147 | 148 | gettext: 149 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 150 | @echo 151 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 152 | 153 | changes: 154 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 155 | @echo 156 | @echo "The overview file is in $(BUILDDIR)/changes." 157 | 158 | linkcheck: 159 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 160 | @echo 161 | @echo "Link check complete; look for any errors in the above output " \ 162 | "or in $(BUILDDIR)/linkcheck/output.txt." 163 | 164 | doctest: 165 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 166 | @echo "Testing of doctests in the sources finished, look at the " \ 167 | "results in $(BUILDDIR)/doctest/output.txt." 168 | 169 | xml: 170 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 171 | @echo 172 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 173 | 174 | pseudoxml: 175 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 176 | @echo 177 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." 178 | -------------------------------------------------------------------------------- /doc/_static/ace_input_wang04.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/partofthething/ace/380fba1394dd93e2140e35f5ba32fd0df2ac91dd/doc/_static/ace_input_wang04.png -------------------------------------------------------------------------------- /doc/_static/ace_transforms_wang04.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/partofthething/ace/380fba1394dd93e2140e35f5ba32fd0df2ac91dd/doc/_static/ace_transforms_wang04.png -------------------------------------------------------------------------------- /doc/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # ace documentation build configuration file, created by 4 | # sphinx-quickstart on Sat Nov 1 14:21:00 2014. 5 | # 6 | # This file is execfile()d with the current directory set to its 7 | # containing dir. 8 | # 9 | # Note that not all possible configuration values are present in this 10 | # autogenerated file. 11 | # 12 | # All configuration values have a default; values that are commented out 13 | # serve to show the default. 14 | 15 | import sys 16 | import os 17 | 18 | import sphinx_rtd_theme 19 | 20 | 21 | # If extensions (or modules to document with autodoc) are in another directory, 22 | # add these directories to sys.path here. If the directory is relative to the 23 | # documentation root, use os.path.abspath to make it absolute, like shown here. 24 | sys.path.insert(0, os.path.abspath('..')) 25 | 26 | # -- General configuration ------------------------------------------------ 27 | 28 | # If your documentation needs a minimal Sphinx version, state it here. 29 | # needs_sphinx = '1.0' 30 | 31 | # Add any Sphinx extension module names here, as strings. They can be 32 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 33 | # ones. 34 | extensions = [ 35 | 'sphinx.ext.autodoc', 36 | 'sphinx.ext.mathjax', 37 | 'sphinx.ext.viewcode', 38 | 'numpydoc', 39 | "sphinx_rtd_theme", 40 | ] 41 | 42 | # Add any paths that contain templates here, relative to this directory. 43 | templates_path = ['_templates'] 44 | 45 | # The suffix of source filenames. 46 | source_suffix = '.rst' 47 | 48 | # The encoding of source files. 49 | # source_encoding = 'utf-8-sig' 50 | 51 | # The master toctree document. 52 | master_doc = 'index' 53 | 54 | # General information about the project. 55 | project = u'ace' 56 | copyright = u'2014-2020, Nick Touran' 57 | 58 | 59 | # The version info for the project you're documenting, acts as replacement for 60 | # |version| and |release|, also used in various other places throughout the 61 | # built documents. 62 | # 63 | # The short X.Y version. 64 | import ace 65 | version = ace.__version__ 66 | # The full version, including alpha/beta/rc tags. 67 | release = ace.__version__ 68 | 69 | # The language for content autogenerated by Sphinx. Refer to documentation 70 | # for a list of supported languages. 71 | # language = None 72 | 73 | # There are two options for replacing |today|: either, you set today to some 74 | # non-false value, then it is used: 75 | # today = '' 76 | # Else, today_fmt is used as the format for a strftime call. 77 | # today_fmt = '%B %d, %Y' 78 | 79 | # List of patterns, relative to source directory, that match files and 80 | # directories to ignore when looking for source files. 81 | exclude_patterns = ['_build'] 82 | 83 | # The reST default role (used for this markup: `text`) to use for all 84 | # documents. 85 | # default_role = None 86 | 87 | # If true, '()' will be appended to :func: etc. cross-reference text. 88 | # add_function_parentheses = True 89 | 90 | # If true, the current module name will be prepended to all description 91 | # unit titles (such as .. function::). 92 | # add_module_names = True 93 | 94 | # If true, sectionauthor and moduleauthor directives will be shown in the 95 | # output. They are ignored by default. 96 | # show_authors = False 97 | 98 | # The name of the Pygments (syntax highlighting) style to use. 99 | pygments_style = 'sphinx' 100 | 101 | # A list of ignored prefixes for module index sorting. 102 | # modindex_common_prefix = [] 103 | 104 | # If true, keep warnings as "system message" paragraphs in the built documents. 105 | # keep_warnings = False 106 | 107 | 108 | # -- Options for HTML output ---------------------------------------------- 109 | 110 | # The theme to use for HTML and HTML Help pages. See the documentation for 111 | # a list of builtin themes. 112 | html_theme = "sphinx_rtd_theme" 113 | 114 | # Theme options are theme-specific and customize the look and feel of a theme 115 | # further. For a list of options available for each theme, see the 116 | # documentation. 117 | # html_theme_options = {} 118 | html_theme_options = { 119 | 'collapse_navigation': False, 120 | 'sticky_navigation': True, 121 | 'navigation_depth': 4, 122 | 'includehidden': True, 123 | } 124 | # Add any paths that contain custom themes here, relative to this directory. 125 | # html_theme_path = [] 126 | 127 | # The name for this set of Sphinx documents. If None, it defaults to 128 | # " v documentation". 129 | # html_title = None 130 | 131 | # A shorter title for the navigation bar. Default is the same as html_title. 132 | # html_short_title = None 133 | 134 | # The name of an image file (relative to this directory) to place at the top 135 | # of the sidebar. 136 | # html_logo = None 137 | 138 | # The name of an image file (within the static path) to use as favicon of the 139 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 140 | # pixels large. 141 | # html_favicon = None 142 | 143 | # Add any paths that contain custom static files (such as style sheets) here, 144 | # relative to this directory. They are copied after the builtin static files, 145 | # so a file named "default.css" will overwrite the builtin "default.css". 146 | html_static_path = ['_static'] 147 | 148 | # Add any extra paths that contain custom files (such as robots.txt or 149 | # .htaccess) here, relative to this directory. These files are copied 150 | # directly to the root of the documentation. 151 | # html_extra_path = [] 152 | 153 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 154 | # using the given strftime format. 155 | # html_last_updated_fmt = '%b %d, %Y' 156 | 157 | # If true, SmartyPants will be used to convert quotes and dashes to 158 | # typographically correct entities. 159 | # html_use_smartypants = True 160 | 161 | # Custom sidebar templates, maps document names to template names. 162 | # html_sidebars = {} 163 | 164 | # Additional templates that should be rendered to pages, maps page names to 165 | # template names. 166 | # html_additional_pages = {} 167 | 168 | # If false, no module index is generated. 169 | # html_domain_indices = True 170 | 171 | # If false, no index is generated. 172 | # html_use_index = True 173 | 174 | # If true, the index is split into individual pages for each letter. 175 | # html_split_index = False 176 | 177 | # If true, links to the reST sources are added to the pages. 178 | # html_show_sourcelink = True 179 | 180 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 181 | # html_show_sphinx = True 182 | 183 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 184 | # html_show_copyright = True 185 | 186 | # If true, an OpenSearch description file will be output, and all pages will 187 | # contain a tag referring to it. The value of this option must be the 188 | # base URL from which the finished HTML is served. 189 | # html_use_opensearch = '' 190 | 191 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 192 | # html_file_suffix = None 193 | 194 | # Output file base name for HTML help builder. 195 | htmlhelp_basename = 'acedoc' 196 | 197 | 198 | # -- Options for LaTeX output --------------------------------------------- 199 | 200 | latex_elements = { 201 | # The paper size ('letterpaper' or 'a4paper'). 202 | # 'papersize': 'letterpaper', 203 | 204 | # The font size ('10pt', '11pt' or '12pt'). 205 | # 'pointsize': '10pt', 206 | 207 | # Additional stuff for the LaTeX preamble. 208 | # 'preamble': '', 209 | } 210 | 211 | # Grouping the document tree into LaTeX files. List of tuples 212 | # (source start file, target name, title, 213 | # author, documentclass [howto, manual, or own class]). 214 | latex_documents = [ 215 | ('index', 'ace.tex', u'ace Documentation', 216 | u'Nick Touran', 'manual'), 217 | ] 218 | 219 | # The name of an image file (relative to this directory) to place at the top of 220 | # the title page. 221 | # latex_logo = None 222 | 223 | # For "manual" documents, if this is true, then toplevel headings are parts, 224 | # not chapters. 225 | # latex_use_parts = False 226 | 227 | # If true, show page references after internal links. 228 | # latex_show_pagerefs = False 229 | 230 | # If true, show URL addresses after external links. 231 | # latex_show_urls = False 232 | 233 | # Documents to append as an appendix to all manuals. 234 | # latex_appendices = [] 235 | 236 | # If false, no module index is generated. 237 | # latex_domain_indices = True 238 | 239 | 240 | # -- Options for manual page output --------------------------------------- 241 | 242 | # One entry per manual page. List of tuples 243 | # (source start file, name, description, authors, manual section). 244 | man_pages = [ 245 | ('index', 'ace', u'ace Documentation', 246 | [u'Nick Touran'], 1) 247 | ] 248 | 249 | # If true, show URL addresses after external links. 250 | # man_show_urls = False 251 | 252 | 253 | # -- Options for Texinfo output ------------------------------------------- 254 | 255 | # Grouping the document tree into Texinfo files. List of tuples 256 | # (source start file, target name, title, author, 257 | # dir menu entry, description, category) 258 | texinfo_documents = [ 259 | ('index', 'ace', u'ace Documentation', 260 | u'Nick Touran', 'ace', 'Alternating Conditional Expectation solver', 261 | 'Miscellaneous'), 262 | ] 263 | 264 | # Documents to append as an appendix to all manuals. 265 | # texinfo_appendices = [] 266 | 267 | # If false, no module index is generated. 268 | # texinfo_domain_indices = True 269 | 270 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 271 | # texinfo_show_urls = 'footnote' 272 | 273 | # If true, do not generate a @detailmenu in the "Top" node's menu. 274 | # texinfo_no_detailmenu = False 275 | 276 | -------------------------------------------------------------------------------- /doc/index.rst: -------------------------------------------------------------------------------- 1 | 2 | .. include:: ../README.rst 3 | 4 | Indices and tables 5 | ------------------ 6 | 7 | * :ref:`genindex` 8 | * :ref:`modindex` 9 | * :ref:`search` 10 | 11 | 12 | Contents 13 | -------- 14 | .. toctree:: 15 | :maxdepth: 2 16 | 17 | index 18 | samples 19 | -------------------------------------------------------------------------------- /doc/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | REM Command file for Sphinx documentation 4 | 5 | if "%SPHINXBUILD%" == "" ( 6 | set SPHINXBUILD=sphinx-build 7 | ) 8 | set BUILDDIR=_build 9 | set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . 10 | set I18NSPHINXOPTS=%SPHINXOPTS% . 11 | if NOT "%PAPER%" == "" ( 12 | set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% 13 | set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% 14 | ) 15 | 16 | if "%1" == "" goto help 17 | 18 | if "%1" == "help" ( 19 | :help 20 | echo.Please use `make ^` where ^ is one of 21 | echo. html to make standalone HTML files 22 | echo. dirhtml to make HTML files named index.html in directories 23 | echo. singlehtml to make a single large HTML file 24 | echo. pickle to make pickle files 25 | echo. json to make JSON files 26 | echo. htmlhelp to make HTML files and a HTML help project 27 | echo. qthelp to make HTML files and a qthelp project 28 | echo. devhelp to make HTML files and a Devhelp project 29 | echo. epub to make an epub 30 | echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter 31 | echo. text to make text files 32 | echo. man to make manual pages 33 | echo. texinfo to make Texinfo files 34 | echo. gettext to make PO message catalogs 35 | echo. changes to make an overview over all changed/added/deprecated items 36 | echo. xml to make Docutils-native XML files 37 | echo. pseudoxml to make pseudoxml-XML files for display purposes 38 | echo. linkcheck to check all external links for integrity 39 | echo. doctest to run all doctests embedded in the documentation if enabled 40 | goto end 41 | ) 42 | 43 | if "%1" == "clean" ( 44 | for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i 45 | del /q /s %BUILDDIR%\* 46 | goto end 47 | ) 48 | 49 | 50 | %SPHINXBUILD% 2> nul 51 | if errorlevel 9009 ( 52 | echo. 53 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 54 | echo.installed, then set the SPHINXBUILD environment variable to point 55 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 56 | echo.may add the Sphinx directory to PATH. 57 | echo. 58 | echo.If you don't have Sphinx installed, grab it from 59 | echo.http://sphinx-doc.org/ 60 | exit /b 1 61 | ) 62 | 63 | if "%1" == "html" ( 64 | %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html 65 | if errorlevel 1 exit /b 1 66 | echo. 67 | echo.Build finished. The HTML pages are in %BUILDDIR%/html. 68 | goto end 69 | ) 70 | 71 | if "%1" == "dirhtml" ( 72 | %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml 73 | if errorlevel 1 exit /b 1 74 | echo. 75 | echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. 76 | goto end 77 | ) 78 | 79 | if "%1" == "singlehtml" ( 80 | %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml 81 | if errorlevel 1 exit /b 1 82 | echo. 83 | echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. 84 | goto end 85 | ) 86 | 87 | if "%1" == "pickle" ( 88 | %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle 89 | if errorlevel 1 exit /b 1 90 | echo. 91 | echo.Build finished; now you can process the pickle files. 92 | goto end 93 | ) 94 | 95 | if "%1" == "json" ( 96 | %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json 97 | if errorlevel 1 exit /b 1 98 | echo. 99 | echo.Build finished; now you can process the JSON files. 100 | goto end 101 | ) 102 | 103 | if "%1" == "htmlhelp" ( 104 | %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp 105 | if errorlevel 1 exit /b 1 106 | echo. 107 | echo.Build finished; now you can run HTML Help Workshop with the ^ 108 | .hhp project file in %BUILDDIR%/htmlhelp. 109 | goto end 110 | ) 111 | 112 | if "%1" == "qthelp" ( 113 | %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp 114 | if errorlevel 1 exit /b 1 115 | echo. 116 | echo.Build finished; now you can run "qcollectiongenerator" with the ^ 117 | .qhcp project file in %BUILDDIR%/qthelp, like this: 118 | echo.^> qcollectiongenerator %BUILDDIR%\qthelp\ace.qhcp 119 | echo.To view the help file: 120 | echo.^> assistant -collectionFile %BUILDDIR%\qthelp\ace.ghc 121 | goto end 122 | ) 123 | 124 | if "%1" == "devhelp" ( 125 | %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp 126 | if errorlevel 1 exit /b 1 127 | echo. 128 | echo.Build finished. 129 | goto end 130 | ) 131 | 132 | if "%1" == "epub" ( 133 | %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub 134 | if errorlevel 1 exit /b 1 135 | echo. 136 | echo.Build finished. The epub file is in %BUILDDIR%/epub. 137 | goto end 138 | ) 139 | 140 | if "%1" == "latex" ( 141 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 142 | if errorlevel 1 exit /b 1 143 | echo. 144 | echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. 145 | goto end 146 | ) 147 | 148 | if "%1" == "latexpdf" ( 149 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 150 | cd %BUILDDIR%/latex 151 | make all-pdf 152 | cd %BUILDDIR%/.. 153 | echo. 154 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 155 | goto end 156 | ) 157 | 158 | if "%1" == "latexpdfja" ( 159 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 160 | cd %BUILDDIR%/latex 161 | make all-pdf-ja 162 | cd %BUILDDIR%/.. 163 | echo. 164 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 165 | goto end 166 | ) 167 | 168 | if "%1" == "text" ( 169 | %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text 170 | if errorlevel 1 exit /b 1 171 | echo. 172 | echo.Build finished. The text files are in %BUILDDIR%/text. 173 | goto end 174 | ) 175 | 176 | if "%1" == "man" ( 177 | %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man 178 | if errorlevel 1 exit /b 1 179 | echo. 180 | echo.Build finished. The manual pages are in %BUILDDIR%/man. 181 | goto end 182 | ) 183 | 184 | if "%1" == "texinfo" ( 185 | %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo 186 | if errorlevel 1 exit /b 1 187 | echo. 188 | echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. 189 | goto end 190 | ) 191 | 192 | if "%1" == "gettext" ( 193 | %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale 194 | if errorlevel 1 exit /b 1 195 | echo. 196 | echo.Build finished. The message catalogs are in %BUILDDIR%/locale. 197 | goto end 198 | ) 199 | 200 | if "%1" == "changes" ( 201 | %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes 202 | if errorlevel 1 exit /b 1 203 | echo. 204 | echo.The overview file is in %BUILDDIR%/changes. 205 | goto end 206 | ) 207 | 208 | if "%1" == "linkcheck" ( 209 | %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck 210 | if errorlevel 1 exit /b 1 211 | echo. 212 | echo.Link check complete; look for any errors in the above output ^ 213 | or in %BUILDDIR%/linkcheck/output.txt. 214 | goto end 215 | ) 216 | 217 | if "%1" == "doctest" ( 218 | %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest 219 | if errorlevel 1 exit /b 1 220 | echo. 221 | echo.Testing of doctests in the sources finished, look at the ^ 222 | results in %BUILDDIR%/doctest/output.txt. 223 | goto end 224 | ) 225 | 226 | if "%1" == "xml" ( 227 | %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml 228 | if errorlevel 1 exit /b 1 229 | echo. 230 | echo.Build finished. The XML files are in %BUILDDIR%/xml. 231 | goto end 232 | ) 233 | 234 | if "%1" == "pseudoxml" ( 235 | %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml 236 | if errorlevel 1 exit /b 1 237 | echo. 238 | echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. 239 | goto end 240 | ) 241 | 242 | :end 243 | -------------------------------------------------------------------------------- /doc/readme.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../README.rst -------------------------------------------------------------------------------- /doc/samples.rst: -------------------------------------------------------------------------------- 1 | 2 | Sample ACE Problems 3 | =================== 4 | Several sample problems from public literature are provided in the :py:mod:`ace.samples` 5 | subpackage. The one from [Wang04]_ is particularly good at demonstrating the power 6 | of the ACE algorithm. It starts by building test data using the following formula: 7 | 8 | .. math:: 9 | 10 | y(X) = \text{log}\left(4 + \text{sin}(4 X_0) + |X_1| + X_2^2 + X_3^3 + X_4 + 0.1\epsilon\right) 11 | 12 | where :math:`\epsilon` is sampled from standard normal distribution. The input data (N=200) is 13 | shown below. As you can see, it's pretty ugly, and it'd be difficult to understand the 14 | underlying structure by doing normal regressions. 15 | 16 | .. image:: _static/ace_input_wang04.png 17 | :alt: Plot of the input data, which is all over the place 18 | 19 | But go ahead and try running ACE on it, like this:: 20 | 21 | from ace.samples import wang04 22 | wang04.run_wang04() 23 | 24 | This will produce resulting transforms that look like the ones in the figure below. As you can 25 | see, ACE performed surprisingly well at extracting the underlying structure. You can clearly 26 | see the sine wave, the absolute value, the cubic, etc. It's fabulous! 27 | 28 | .. image:: _static/ace_transforms_wang04.png 29 | :alt: Plot of the output transforms, which clearly show the underlying structure 30 | -------------------------------------------------------------------------------- /profile_ace.py: -------------------------------------------------------------------------------- 1 | """ 2 | Profile ACE. 3 | 4 | Plot with a command like this: 5 | ../../../gprof2dot/gprof2dot.py -f pstats ace.stats | dot -Tpng -o ace_profile.png 6 | """ 7 | import cProfile 8 | 9 | import ace.ace 10 | import ace.validation.sample_problems 11 | 12 | if __name__ == '__main__': 13 | x, y = ace.validation.sample_problems.sample_ace_problem_wang04(N=200) 14 | ace_solver = ace.ace.ACESolver() 15 | ace_solver.specify_data_set(x, y) 16 | cProfile.run('ace_solver.solve()', 'ace.stats') 17 | 18 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | numpy>=1.0 2 | scipy>=0.12 3 | numpydoc>=0.1 4 | matplotlib>=1.0 5 | -------------------------------------------------------------------------------- /requirements_test.txt: -------------------------------------------------------------------------------- 1 | pylint 2 | pydocstyle 3 | coveralls>=1.1 4 | pytest>=2.9.2 5 | pytest-cov>=2.3.1 6 | pytest-cov>=2.3.1 7 | pytest-timeout>=1.2.0 8 | pytest-sugar>=0.7.1 9 | sphinx-rtd-theme 10 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | 3 | with open('README.rst') as f: 4 | long_description = f.read() 5 | 6 | setup( 7 | name='ace', 8 | version='0.3.3', 9 | description= 10 | 'Non-parametric multivariate regressions by Alternating Conditional Expectations', 11 | author='Nick Touran', 12 | author_email='ace@partofthething.com', 13 | url='https://github.com/partofthething/ace', 14 | packages=find_packages(), 15 | license='MIT', 16 | long_description=long_description, 17 | install_requires=['numpy', 'scipy>=0.17'], 18 | keywords='regression ace multivariate statistics', 19 | classifiers=[ 20 | 'Development Status :: 3 - Alpha', 21 | 'Intended Audience :: Science/Research', 22 | 'Topic :: Scientific/Engineering :: Information Analysis', 23 | 'License :: OSI Approved :: MIT License', 24 | 'Programming Language :: Python :: 2', 25 | 'Programming Language :: Python :: 2.7', 26 | 'Programming Language :: Python :: 3', 27 | 'Programming Language :: Python :: 3.4', 28 | 'Programming Language :: Python :: 3.5', 29 | 'Programming Language :: Python :: 3.6', 30 | 'Programming Language :: Python :: 3.7', 31 | 'Programming Language :: Python :: 3.8', 32 | 'Programming Language :: Python :: 3.9', 33 | 'Programming Language :: Python :: 3.10', 34 | 'Programming Language :: Python :: 3.11', 35 | ], 36 | test_suite='tests') 37 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | # content of: tox.ini , put in same dir as setup.py 2 | [tox] 3 | envlist = py27,py36,lint 4 | [testenv] 5 | deps= 6 | -r{toxinidir}/requirements.txt 7 | -r{toxinidir}/requirements_test.txt 8 | setenv = 9 | PYTHONPATH = {toxinidir}:{toxinidir}/ace 10 | commands = 11 | py.test 12 | 13 | [testenv:lint] 14 | basepython = python3 15 | ignore_errors = True 16 | commands = 17 | pylint ace 18 | pydocstyle ace tests 19 | --------------------------------------------------------------------------------