├── .gitignore ├── README.md ├── data ├── _resp.txt ├── _synt3elbo.txt ├── faithful.txt ├── gaussian.txt └── img │ ├── graphical_models.png │ └── results_fullGMM_OF.png ├── demo.py ├── src ├── __init__.py ├── base.py ├── viGMM_CB.py └── viGMM_full.py ├── utils.py ├── viGMM_CB.ipynb └── viGMM_McGrory.ipynb /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__/ 2 | */__pycache__/ 3 | .ipynb_checkpoints/ 4 | */.ipynb_checkpoints/ 5 | 6 | .vscode 7 | 8 | test.py -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Variational Bayesian Model Selection for Mixture Distributions 2 | 3 | Two Variational Bayesian Gaussian mixture models proposed in [1] and [2] are compared. Their graphical models are respectively shown below. 4 | ![graphical models](data/img/graphical_models.png) 5 | - The implementation for the first model [1], is implemented through the class `VariationalGaussianMixture` in `src.viGMM_CB`. 6 | - The full Bayesian GMM model [2] is implemented in the class `VariationalGaussianMixture` in `src.viGMM_full`. 7 | ## Installation 8 | First clone the project 9 | ``` 10 | git clone https://github.com/JulienNonin/variational-gaussian-mixture.git 11 | cd variational-gaussian-mixture 12 | ``` 13 | `variational-gaussian-mixture` runs on Python 3.7. and only requires `matplotlib ≥ 3.2.1` and `numpy ≥ 1.18.1`. 14 | 15 | ## Usage 16 | Import the module 17 | ``` 18 | import src as mixture 19 | ``` 20 | Load the "Old Faithful" data set and standardize the data. 21 | ``` 22 | X = np.loadtxt('data/faithful.txt') 23 | X = (X - X.mean(axis=0)) / X.std(axis=0) 24 | ``` 25 | Apply a Bayesian GMM (`mixture.VariationalGaussianMixtureCB` or `mixture.VariationalGaussianMixture`) 26 | ``` 27 | model = mixture.VariationalGaussianMixtureCB(K=10, display=True, max_iter=201, plot_period=200, init_param="kmeans") 28 | model.fit(X) 29 | ``` 30 | This should produce the following output 31 | ![results](data/img/results_fullGMM_OF.png) 32 | 33 | ## References 34 | [1] A. Corduneanu and C. Bishop, *Variational Bayesian Model Selection for Mixture Distributions*. in *Proc. AI 35 | and Statistics Conf.*, Jan. 2001, pp. 27-34. 36 | 37 | [2] C. Bishop, *Pattern Recognition and Machine Learning (Information Science and Statistics)*. New York: 38 | Springer-Verlag, 2006. 39 | 40 | ---- 41 | ## To do 42 | 43 | - [x] Create this README 44 | - [x] Finally make the model without prior on mixing coefficients (viGMM_CB) work 45 | - [x] Create a parent class (BaseGaussianMixture) 46 | - [ ] move fit_predict to the parent class 47 | - [ ] Generate a set of synthetic data 48 | - [ ] Fix random initialization of responsabilities 49 | - [x] Compute the ELBO for GMM_CB 50 | - [x] Compute the ELBO for GMM_full 51 | - [x] Better visualization of GM 52 | - [ ] Stopping criterion using the ELBO 53 | - [ ] Make prediction 54 | - [ ] Add prior on mean for GMM_CB to allow non-standardization -------------------------------------------------------------------------------- /data/_resp.txt: -------------------------------------------------------------------------------- 1 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 2 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 3 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 4 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 5 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 6 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 7 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 8 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 9 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 10 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 11 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 12 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 13 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 14 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 15 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 16 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 17 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 18 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 19 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 20 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 21 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 22 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 23 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 24 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 25 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 26 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 27 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 28 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 29 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 30 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 31 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 32 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 33 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 34 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 35 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 36 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 37 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 38 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 39 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 40 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 41 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 42 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 43 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 44 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 45 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 46 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 47 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 48 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 49 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 50 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 51 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 52 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 53 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 54 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 55 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 56 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 57 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 58 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 59 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 60 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 61 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 62 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 63 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 64 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 65 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 66 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 67 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 68 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 69 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 70 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 71 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 72 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 73 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 74 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 75 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 76 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 77 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 78 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 79 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 80 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 81 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 82 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 83 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 84 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 85 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 86 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 87 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 88 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 89 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 90 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 91 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 92 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 93 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 94 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 95 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 96 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 97 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 98 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 99 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 100 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 101 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 102 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 103 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 104 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 105 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 106 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 107 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 108 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 109 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 110 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 111 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 112 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 113 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 114 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 115 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 116 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 117 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 118 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 119 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 120 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 121 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 122 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 123 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 124 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 125 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 126 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 127 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 128 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 129 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 130 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 131 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 132 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 133 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 134 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 135 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 136 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 137 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 138 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 139 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 140 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 141 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 142 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 143 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 144 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 145 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 146 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 147 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 148 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 149 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 150 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 151 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 152 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 153 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 154 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 155 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 156 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 157 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 158 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 159 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 160 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 161 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 162 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 163 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 164 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 165 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 166 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 167 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 168 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 169 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 170 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 171 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 172 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 173 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 174 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 175 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 176 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 177 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 178 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 179 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 180 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 181 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 182 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 183 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 184 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 185 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 186 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 187 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 188 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 189 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 190 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 191 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 192 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 193 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 194 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 195 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 196 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 197 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 198 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 199 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 200 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 201 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 202 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 203 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 204 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 205 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 206 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 207 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 208 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 209 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 210 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 211 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 212 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 213 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 214 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 215 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 216 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 217 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 218 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 219 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 220 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 221 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 222 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 223 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 224 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 225 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 226 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 227 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 228 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 229 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 230 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 231 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 232 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 233 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 234 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 235 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 236 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 237 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 238 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 239 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 240 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 241 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 242 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 243 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 244 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 245 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 246 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 247 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 248 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 249 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 250 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 251 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 252 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 253 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 254 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 255 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 256 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 257 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 258 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 259 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 260 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 261 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 262 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 263 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 264 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 265 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 266 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 267 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 268 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 269 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 270 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 271 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 272 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 273 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 274 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 275 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 276 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 277 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 278 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 279 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 280 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 281 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 282 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 283 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 284 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 285 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 286 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 287 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 288 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 289 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 290 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 291 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 292 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 293 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 294 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 295 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 296 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 297 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 298 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 299 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 300 | 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 301 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 302 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 303 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 304 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 305 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 306 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 307 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 308 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 309 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 310 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 311 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 312 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 313 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 314 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 315 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 316 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 317 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 318 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 319 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 320 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 321 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 322 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 323 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 324 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 325 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 326 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 327 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 328 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 329 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 330 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 331 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 332 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 333 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 334 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 335 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 336 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 337 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 338 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 339 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 340 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 341 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 342 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 343 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 344 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 345 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 346 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 347 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 348 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 349 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 350 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 351 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 352 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 353 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 354 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 355 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 356 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 357 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 358 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 359 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 360 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 361 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 362 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 363 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 364 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 365 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 366 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 367 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 368 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 369 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 370 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 371 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 372 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 373 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 374 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 375 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 376 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 377 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 378 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 379 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 380 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 381 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 382 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 383 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 384 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 385 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 386 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 387 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 388 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 389 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 390 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 391 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 392 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 393 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 394 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 395 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 396 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 397 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 398 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 399 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 400 | 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 401 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 402 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 403 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 404 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 405 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 406 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 407 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 408 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 409 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 410 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 411 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 412 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 413 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 414 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 415 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 416 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 417 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 418 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 419 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 420 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 421 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 422 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 423 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 424 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 425 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 426 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 427 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 428 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 429 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 430 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 431 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 432 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 433 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 434 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 435 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 436 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 437 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 438 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 439 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 440 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 441 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 442 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 443 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 444 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 445 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 446 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 447 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 448 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 449 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 450 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 451 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 452 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 453 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 454 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 455 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 456 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 457 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 458 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 459 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 460 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 461 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 462 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 463 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 464 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 465 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 466 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 467 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 468 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 469 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 470 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 471 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 472 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 473 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 474 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 475 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 476 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 477 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 478 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 479 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 480 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 481 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 482 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 483 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 484 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 485 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 486 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 487 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 488 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 489 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 490 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 491 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 492 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 493 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 494 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 495 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 496 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 497 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 498 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 499 | 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 500 | 0.000000000000000000e+00 1.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 0.000000000000000000e+00 501 | -------------------------------------------------------------------------------- /data/_synt3elbo.txt: -------------------------------------------------------------------------------- 1 | -2.884685034157791872e+00 5.514795717002801689e-01 2 | 3.299074949567143644e+00 7.046580104669426525e-01 3 | 2.047595719131244607e+00 -4.444740987825747114e-01 4 | 1.111572346898581420e+00 3.133045759357945825e-01 5 | -7.885673964973608507e-01 7.413997357526145304e-01 6 | -8.992126976511600756e-01 1.579757732446698815e-01 7 | 1.231354362703589000e+00 4.913873226266997074e-01 8 | 1.204589544411871715e+00 -3.146620314210348823e-01 9 | -5.168804092342502154e-02 4.123923498413925581e-01 10 | -1.161344329949644205e+00 -1.042864668909786741e-01 11 | 8.593816581979651925e-01 -5.056387253402609971e-01 12 | -3.467682132349738100e-01 -1.014781858606348175e+00 13 | 1.089128209680312320e+00 3.779606926830774150e-01 14 | 1.850515135012533952e+00 1.017462246433947226e-01 15 | -1.947731820645047351e+00 4.723387249771849317e-01 16 | -2.243333976540256902e+00 1.270788797237586643e+00 17 | -1.034188349258336054e-01 2.720365910756082362e-01 18 | 5.816749121184171256e-02 1.322903926421718390e+00 19 | -6.633606121592788840e-01 -1.702752314819780677e-01 20 | 6.541748748786625767e-02 1.250611023331695271e-01 21 | 2.925752121486962842e+00 3.412521162783663797e-01 22 | -1.047642839827499062e+00 2.373468422414008738e-01 23 | -2.241391757874152013e+00 3.356476015584635569e-02 24 | -1.069684488178263448e+00 -1.005202262608630992e+00 25 | 1.365196648598077322e+00 5.376306926611464920e-01 26 | 7.880417980430456382e-01 2.450205285337738181e-01 27 | -6.577100852809691922e-01 1.195366015574148982e-01 28 | -1.431262163951863897e+00 6.191965439548644534e-01 29 | -3.227569444888703232e-01 5.997380648048464824e-01 30 | 2.764598934991153723e-01 3.785880166714751249e-01 31 | -2.703605284989178514e+00 4.057431128706991963e-01 32 | 1.213078676483583918e+00 5.665159030066975188e-01 33 | 1.029000791674668791e-01 -9.380541645469882006e-01 34 | 1.704013366260866524e+00 -2.233792462600893980e-01 35 | -9.095098990484849766e-02 7.910051035949041687e-01 36 | 2.372384529471460168e-01 -2.917335445860909648e-01 37 | 1.570181541733521591e-02 1.000877957786768579e+00 38 | 3.476693624889101719e-01 -9.093348397479741330e-01 39 | 1.323822051430607161e+00 -4.119025510842374595e-03 40 | -2.037237829435994296e+00 -2.245004033885580896e-01 41 | 2.099827854623812051e+00 -2.502269847698028182e-01 42 | -3.324183990655342757e-01 8.316749288521926853e-02 43 | -2.971455608086887579e-01 4.780197032299192683e-01 44 | -1.061944469156343107e+00 8.549307387941533498e-02 45 | -7.759996383275880039e-01 4.609118555802888539e-01 46 | -1.101918596750431689e+00 -3.640240008718281794e-01 47 | 3.196176758478370683e-01 -4.014422978009014598e-01 48 | -1.917210126854139052e+00 -6.588977119962056983e-02 49 | -2.984379561991932661e+00 6.021294722827202905e-01 50 | -1.679821063374403733e+00 -3.565718350572247286e-01 51 | 4.646423940221627857e-02 -3.422996708573894931e-01 52 | 1.169880275962150301e-01 7.176635702547662365e-01 53 | 3.605299266252095514e+00 -1.086694869588605561e-01 54 | 1.989694623739948653e+00 -5.739611841193337227e-02 55 | 1.265309504552551090e+00 1.201540064827681720e+00 56 | 4.886955433768137569e-01 1.739229386824505119e-01 57 | -2.001539307137782497e+00 -1.810734731322057589e-01 58 | -5.855669316808799207e-01 3.462391171189976857e-01 59 | 6.805565643520036789e-02 2.302550518295181925e-01 60 | 1.707581534826402070e+00 3.162882033237944812e-01 61 | 1.411259253008525727e+00 -1.017455948215043149e+00 62 | -1.005804762294325583e+00 8.133441196949769969e-02 63 | 1.187633588113806260e+00 8.177960055745350787e-03 64 | 9.024767912947588844e-01 -3.209461943786864196e-02 65 | 3.862922325753053165e-01 -4.139383894414004339e-01 66 | -4.439543071880311942e-01 -8.344302187572564256e-01 67 | -4.969728330019012597e-01 -1.189521987629653110e-01 68 | -5.711969860517064346e-01 -5.523038286250455675e-01 69 | 5.320583502695327427e-01 -5.742263805995461573e-02 70 | 1.297845086410187365e+00 -1.763654107560392870e-01 71 | -8.302901358119302300e-01 4.911519576830846168e-01 72 | -1.336862305171648835e+00 -3.327225433846304758e-01 73 | -1.498538660469272310e+00 3.076271241500069809e-01 74 | 2.466778673382744591e-01 8.614604713299348071e-03 75 | 1.545735571910520356e+00 4.031802248750771533e-01 76 | -1.672593430900310985e+00 -1.271495456955755943e+00 77 | -7.521807496657001435e-02 1.673013390218419383e-01 78 | 3.200592325181472741e-01 -6.498710293548746186e-01 79 | -2.148461798639500486e+00 6.220986154501207954e-01 80 | -2.715542802415015888e+00 8.926781571805380011e-01 81 | 3.047133642199068571e-01 5.213455301874174719e-01 82 | -3.349352712685111366e-01 5.134629286406084958e-01 83 | 2.598903372643157628e-01 8.788078092068925262e-02 84 | 2.235848295344968317e+00 5.297783574251525157e-02 85 | -4.055688538994263737e-01 7.748867654284027906e-02 86 | 3.170811747321491403e+00 5.396342182327652504e-01 87 | -5.820956963452232902e-01 8.289150944620241601e-01 88 | -1.340718942599880670e+00 1.385705413362332272e-01 89 | 2.722830464558682606e-01 4.045022670178308744e-01 90 | 4.763641899797438350e-01 1.141883946410466588e-01 91 | -1.103670114527298463e+00 5.148355717960276801e-01 92 | 6.756876714623345492e-01 6.621300559960923149e-01 93 | 2.104028253267371529e-01 -4.078794543617802471e-01 94 | -4.780068357486739750e-01 -6.025068317892365855e-01 95 | 6.426182666559371315e-01 7.116139982291076471e-01 96 | 1.106439016582346557e+00 -1.979931948836189948e-01 97 | 2.118278666536863142e+00 -5.276700504749421983e-01 98 | 1.023859906556765509e+00 8.382952433667469894e-01 99 | 5.376827564129902370e-01 1.117327372725801793e-01 100 | -1.695145258627067442e+00 4.280468589531250534e-01 101 | 3.854184913788970268e-02 -6.431706142610147392e-01 102 | 1.638157590835664229e-02 -9.050828811529209039e-01 103 | -1.730247461660683606e-01 -6.551864629363537418e-01 104 | 2.446692603775993055e-02 -7.123317330158569494e-01 105 | 1.848386415831239638e-01 -2.467471579767762435e+00 106 | -4.478225569906000604e-03 2.241401700090591920e+00 107 | -9.488547844975370182e-02 1.514052839644859372e-01 108 | -3.606491306133541297e-01 3.204056301927460382e+00 109 | -8.028183485974144107e-02 -2.860410949926415647e+00 110 | -1.925981602558246852e-01 -1.304819009184055023e+00 111 | 3.068304828689378261e-01 -2.518368259724098834e+00 112 | -2.164294082950401696e-02 -9.594258534366781888e-01 113 | 1.598506865271889232e-01 -1.710564265893432800e+00 114 | -1.259244238264332037e-02 1.221364805434991707e+00 115 | -1.129651511316221964e-03 1.896583104360434202e+00 116 | -6.907214681461631978e-02 -7.047211212889398224e-01 117 | 2.899823015522213221e-01 -5.744141823975259520e-01 118 | 6.425690951640701531e-02 1.201264209282422835e+00 119 | -5.944649670432039512e-02 -1.960210659413814005e+00 120 | -8.231384074992646738e-02 -3.176665707081617329e-01 121 | -2.746993108159399566e-01 1.116541748428316216e+00 122 | 1.237451382489074825e-01 -3.078619044433174246e+00 123 | -6.214580340621441573e-02 -2.075529386857562830e+00 124 | -9.022841544093792784e-03 6.233290999616651895e-01 125 | -1.169799336696196734e-01 -5.178523118226837640e-01 126 | -2.567673408317397987e-01 1.155907458118940134e+00 127 | -4.871804221421242032e-01 9.488704659859649171e-01 128 | 2.848369551780884357e-01 -5.859918264185415282e-01 129 | -1.782212559091772286e-01 -1.348126672121372938e+00 130 | -3.527598417111997842e-01 -1.101467707949032038e+00 131 | -2.971667846688725856e-01 -2.102380990235310498e-01 132 | 2.564947614790651434e-02 -1.457535398695058715e-01 133 | 2.279096059257516294e-01 -1.776084837920392623e+00 134 | 4.463128128090000646e-03 -8.372122908539216812e-01 135 | -8.373371232130891184e-02 5.051491474803659898e-01 136 | 1.264377699322958293e-01 -2.676528421613737940e+00 137 | -3.676343480790100204e-01 1.963107316244899980e+00 138 | 1.625526779010936784e-01 -5.921962736141631600e-01 139 | -3.449777167330694283e-01 -4.424819510484465090e-02 140 | -1.681640700740792416e-01 1.566769518210112677e+00 141 | 2.437896277005300316e-01 -1.435732547600691911e+00 142 | 7.946657252245752090e-02 -1.237122306929850701e+00 143 | 8.362934731706133870e-03 1.045049777470432817e+00 144 | -6.591718895496806452e-02 -8.688637620611751045e-01 145 | 4.687173254708061343e-01 -7.373674365758691174e-01 146 | 3.441454357555820409e-02 -4.683795631108484958e-01 147 | -2.297536288933109683e-01 1.079251957642607618e+00 148 | -1.605554980740318827e-01 -1.802384343104726172e+00 149 | 5.829869226479816868e-01 -2.443989411867782735e+00 150 | 2.613700394394785631e-01 -9.417915488687811365e-01 151 | -6.557159899602765796e-02 5.826213347417320554e-01 152 | 2.209153849636087408e-01 -1.616737506845757233e+00 153 | -4.292162075043198244e-01 1.714734104750781585e+00 154 | -2.080522440902149162e-01 2.385477176243036013e+00 155 | 3.113421082955559749e-01 3.406361440408937624e-01 156 | 5.937122541149784938e-02 -1.540409377577331762e+00 157 | -1.804958693461902741e-01 -2.459607247871326008e-02 158 | 6.465757174748917868e-02 4.819365407641559851e-01 159 | -2.215360141360949442e-01 9.701833738386469452e-01 160 | 1.021112763841087318e-01 -8.740120585300612532e-01 161 | 6.883470986223599875e-02 -3.723308140642638681e-02 162 | -3.398779477397175230e-01 1.840479235597283214e+00 163 | -1.680723158567638820e-01 -2.680819391770977056e+00 164 | 3.642638769732324810e-01 -3.131679088588849025e-01 165 | -3.118176992017968607e-01 -2.174012692227911714e+00 166 | -2.267224799021621828e-01 5.854767852673502526e-01 167 | 4.437397749499326360e-02 -6.970332616865179132e-01 168 | -1.194010454352910089e-01 -2.248647492048631014e-01 169 | -6.006642271126539923e-01 1.862010264309371443e+00 170 | -2.065764103339132879e-02 1.804088232171768835e-01 171 | -4.589146548217329369e-01 1.067928366535126683e+00 172 | 2.506425977376490824e-02 4.551709104939061779e-01 173 | -2.072024024211308324e-01 2.641219342055076247e+00 174 | 9.113810938465925771e-02 1.961942460262947163e+00 175 | -4.419858680071243606e-01 -9.550948290772638405e-01 176 | 1.082134039483212407e-01 -1.197014585439566403e+00 177 | -6.532343814668864490e-02 9.621404806508127017e-01 178 | -2.320841795331093083e-01 -1.257528453297619654e-03 179 | -3.934198653021058201e-01 4.500925245798149676e-01 180 | -2.925881605505574745e-01 6.477889849467330929e-01 181 | -6.658893573068573601e-02 -1.998532949789802726e-01 182 | -2.210314167843195343e-01 1.761209993791958306e+00 183 | 9.663313900023634551e-02 -4.774141877114255750e-01 184 | -1.643165844226199268e-01 -4.341596174071809533e-01 185 | -3.145874354843493093e-01 3.353218076172104767e-01 186 | 5.059529078083267739e-02 -5.935020456499245922e-01 187 | 2.240601490545577335e-01 5.260121801418900234e-02 188 | 3.696776576061359676e-01 -1.471417614996942680e+00 189 | 3.848760957724955967e-01 -5.458807335377305892e-01 190 | -8.570255646005481287e-02 -1.135048781269763829e+00 191 | -2.825045452428824322e-01 7.241934368265288968e-02 192 | -1.158146727389608444e-01 -1.762663128139466961e-01 193 | 3.587411685863506339e-02 2.205411521858628099e+00 194 | 3.732435883210369343e-02 7.929257222835597307e-01 195 | 2.087204431430438600e-01 -1.064252247311927801e+00 196 | -3.579353352588577342e-01 5.288581100038454164e-01 197 | -1.650487207022824643e-01 -6.414669066105834760e-01 198 | -1.723315068412643869e-02 -1.283646478939785451e+00 199 | -1.882674132484541685e-01 2.387933104312617516e+00 200 | 5.480845464026937730e-01 -2.792927567069992012e+00 201 | -5.245753669225199367e-01 -9.113934935832456175e-01 202 | 4.954299972842533817e-02 -1.157201613577296950e-01 203 | -6.190432172602714500e-01 -1.451767376995418135e-01 204 | 2.217750488457507496e-01 -3.640834072976556479e-01 205 | -5.411004543775838060e-01 1.398720195120538023e-01 206 | -2.085734515798357336e+00 -1.731006594241705221e+00 207 | 6.996586063106753350e-01 6.963801642940756764e-01 208 | 7.926626127674289979e-01 5.069350713772614592e-01 209 | 2.104591529350614554e-01 -9.684462438035347509e-02 210 | 3.806779514362972905e-01 2.046508415693017724e-01 211 | -1.398173802383273490e-02 3.553874235734372222e-01 212 | 1.133102903190389865e+00 1.035385336448219951e+00 213 | -2.194088891869473645e+00 -3.466832632727542829e-01 214 | -3.508242097997286701e-01 -3.352694485352461506e-01 215 | 2.966765826393401584e-01 3.323749465658294433e-01 216 | 1.413400567362103066e+00 1.487875191446303091e+00 217 | 2.155347898247245986e-01 -4.743176693316330517e-01 218 | -1.466338749773893058e-01 6.777530492793994998e-01 219 | 2.892584983839882606e-01 -2.518070217927347865e-01 220 | -1.626359485053175957e+00 -2.489768455803974767e+00 221 | 1.975860989850926508e+00 1.732251759819176451e+00 222 | 1.083652127014856026e+00 3.023305997491079200e-01 223 | 3.012518313025362438e-01 -6.181640140165194364e-01 224 | 1.527844159794581547e+00 1.251500511323805798e+00 225 | -1.513190984777105452e+00 -1.404341287073078037e+00 226 | -7.569468712769118479e-01 -4.270171704693917492e-02 227 | -1.268635709525240518e+00 -5.840711689885437208e-01 228 | 9.723833124399671179e-01 5.828466957147465655e-01 229 | 1.284424730431897865e-01 4.828345173983470962e-01 230 | 1.462131558034105439e-01 -3.432571885088190888e-01 231 | 1.738912197299276929e+00 9.987941492789134124e-01 232 | -7.616396740508020180e-01 -3.572566072963073713e-01 233 | -4.403311463524612890e-01 3.077581331496710981e-01 234 | 2.001705903100768680e+00 1.439902613995414749e+00 235 | -3.010506460063282486e-01 -6.698095295628101464e-01 236 | 1.211811830720016303e+00 1.194714747071278449e+00 237 | 5.683391505759771700e-01 3.758239610172008915e-01 238 | 8.170691494919968267e-01 6.259901936599868710e-01 239 | 2.225594573737134763e+00 2.038700546249562429e+00 240 | 1.328803066236130259e-01 -3.485457900465398495e-01 241 | -8.749227887252286617e-01 -7.303750931710173422e-01 242 | -8.403335402548678390e-01 -5.853881012748690660e-01 243 | -1.018904457651190976e+00 -1.138581069855793215e+00 244 | -1.436941354572605301e+00 -1.551547067466582774e+00 245 | 6.974743954803410695e-01 5.674013661598522607e-01 246 | -1.030494461143587115e+00 1.528422444201387176e-01 247 | -1.673782342991215311e+00 -4.177079966274191425e-01 248 | 9.825333530183535391e-01 -6.301695302979211377e-02 249 | 2.599877793731517062e-01 2.620135613780715245e-01 250 | -9.426026490537624536e-01 4.309143016976410001e-01 251 | 2.930656482696100684e-01 -4.756859239834067971e-02 252 | -3.900796461324943509e-01 1.103104708501019375e-02 253 | 5.425491756306580760e-01 1.560612683607890683e-01 254 | 1.358686144464091416e+00 5.429102261072110691e-01 255 | 5.956097980894708988e-01 8.427617025564000386e-01 256 | -2.215422827196387723e-01 1.139131552126546820e+00 257 | 9.844325802655268243e-01 5.769558499820740538e-01 258 | -7.442685412581881987e-01 -6.461654023898695254e-01 259 | 6.122745411584779029e-01 -3.236881020478917237e-01 260 | 1.446965378968737337e+00 5.442383544843825138e-01 261 | 1.312218885739200758e+00 1.872008267283950200e+00 262 | -7.090458888906044921e-01 -4.646456314630821594e-03 263 | 6.759097006096358440e-01 -1.497352721811252563e-01 264 | 4.025671350114294333e-01 -3.962640975608971994e-01 265 | 2.219451750723680228e-01 7.427111335069510600e-01 266 | -4.324023001944102829e-01 3.216177446296323850e-01 267 | -1.411620865055578100e-01 8.457786093740184141e-02 268 | -1.309892725670666891e-01 -4.217854534056543203e-01 269 | 1.121878374389372879e-01 -3.807896033914086042e-01 270 | -2.692650484967850311e+00 -1.109691109416084176e+00 271 | -1.085845767276921947e+00 -1.181157336128592972e+00 272 | -1.426508429767634478e+00 -9.838267405709574254e-01 273 | 4.237788573365317557e-01 -7.793205711303384520e-02 274 | -9.788655801789983846e-01 -2.048532452562938644e-01 275 | 1.429819825339970207e-01 -5.729293560281293329e-01 276 | 1.768850956810298136e+00 1.640868718935654114e+00 277 | -1.067143673720562624e+00 -6.469792365923738853e-01 278 | 7.086907824735487749e-01 8.069619086477265724e-02 279 | -7.300540151487714402e-01 -1.261679921510098890e+00 280 | 1.366164815825654966e+00 2.005789420874610318e+00 281 | -3.488973572176452126e-02 -4.549870993901911770e-02 282 | -5.562661306693800212e-01 -2.961134174214256620e-01 283 | -5.047943127612742886e-01 -4.082545756863776854e-01 284 | 2.453105036968950881e+00 1.479716982371783507e+00 285 | 2.126028137061136605e-03 1.773904184575862686e-01 286 | 3.661474790323975825e-01 1.269776331049748075e+00 287 | 6.565280595286178134e-01 2.049547498313175620e+00 288 | -2.246966608919706565e-01 1.624394027084714587e-01 289 | 6.160045900567193700e-01 4.816296644660628679e-01 290 | -3.448411520951377707e-01 -8.170504734088110954e-01 291 | -1.215163874549695100e+00 -8.479824431823052011e-01 292 | 9.168896905793473273e-01 2.250252353400261918e-01 293 | -4.977971466621157748e-01 -6.385190088041809986e-01 294 | -6.918484216650050422e-01 -7.355506290016818882e-01 295 | 4.211557050686507431e-01 1.297454501259335480e+00 296 | -8.329350645540121167e-01 -1.671396578554650058e-03 297 | -3.583079681291707203e-02 7.003436039888639997e-02 298 | 4.192057771608919681e-01 1.404909172281701091e+00 299 | -1.892063869329845938e+00 -1.865591152524552987e+00 300 | 1.602518827899649667e-01 6.094636990727558823e-02 301 | -------------------------------------------------------------------------------- /data/faithful.txt: -------------------------------------------------------------------------------- 1 | 3.600000 79.000000 2 | 1.800000 54.000000 3 | 3.333000 74.000000 4 | 2.283000 62.000000 5 | 4.533000 85.000000 6 | 2.883000 55.000000 7 | 4.700000 88.000000 8 | 3.600000 85.000000 9 | 1.950000 51.000000 10 | 4.350000 85.000000 11 | 1.833000 54.000000 12 | 3.917000 84.000000 13 | 4.200000 78.000000 14 | 1.750000 47.000000 15 | 4.700000 83.000000 16 | 2.167000 52.000000 17 | 1.750000 62.000000 18 | 4.800000 84.000000 19 | 1.600000 52.000000 20 | 4.250000 79.000000 21 | 1.800000 51.000000 22 | 1.750000 47.000000 23 | 3.450000 78.000000 24 | 3.067000 69.000000 25 | 4.533000 74.000000 26 | 3.600000 83.000000 27 | 1.967000 55.000000 28 | 4.083000 76.000000 29 | 3.850000 78.000000 30 | 4.433000 79.000000 31 | 4.300000 73.000000 32 | 4.467000 77.000000 33 | 3.367000 66.000000 34 | 4.033000 80.000000 35 | 3.833000 74.000000 36 | 2.017000 52.000000 37 | 1.867000 48.000000 38 | 4.833000 80.000000 39 | 1.833000 59.000000 40 | 4.783000 90.000000 41 | 4.350000 80.000000 42 | 1.883000 58.000000 43 | 4.567000 84.000000 44 | 1.750000 58.000000 45 | 4.533000 73.000000 46 | 3.317000 83.000000 47 | 3.833000 64.000000 48 | 2.100000 53.000000 49 | 4.633000 82.000000 50 | 2.000000 59.000000 51 | 4.800000 75.000000 52 | 4.716000 90.000000 53 | 1.833000 54.000000 54 | 4.833000 80.000000 55 | 1.733000 54.000000 56 | 4.883000 83.000000 57 | 3.717000 71.000000 58 | 1.667000 64.000000 59 | 4.567000 77.000000 60 | 4.317000 81.000000 61 | 2.233000 59.000000 62 | 4.500000 84.000000 63 | 1.750000 48.000000 64 | 4.800000 82.000000 65 | 1.817000 60.000000 66 | 4.400000 92.000000 67 | 4.167000 78.000000 68 | 4.700000 78.000000 69 | 2.067000 65.000000 70 | 4.700000 73.000000 71 | 4.033000 82.000000 72 | 1.967000 56.000000 73 | 4.500000 79.000000 74 | 4.000000 71.000000 75 | 1.983000 62.000000 76 | 5.067000 76.000000 77 | 2.017000 60.000000 78 | 4.567000 78.000000 79 | 3.883000 76.000000 80 | 3.600000 83.000000 81 | 4.133000 75.000000 82 | 4.333000 82.000000 83 | 4.100000 70.000000 84 | 2.633000 65.000000 85 | 4.067000 73.000000 86 | 4.933000 88.000000 87 | 3.950000 76.000000 88 | 4.517000 80.000000 89 | 2.167000 48.000000 90 | 4.000000 86.000000 91 | 2.200000 60.000000 92 | 4.333000 90.000000 93 | 1.867000 50.000000 94 | 4.817000 78.000000 95 | 1.833000 63.000000 96 | 4.300000 72.000000 97 | 4.667000 84.000000 98 | 3.750000 75.000000 99 | 1.867000 51.000000 100 | 4.900000 82.000000 101 | 2.483000 62.000000 102 | 4.367000 88.000000 103 | 2.100000 49.000000 104 | 4.500000 83.000000 105 | 4.050000 81.000000 106 | 1.867000 47.000000 107 | 4.700000 84.000000 108 | 1.783000 52.000000 109 | 4.850000 86.000000 110 | 3.683000 81.000000 111 | 4.733000 75.000000 112 | 2.300000 59.000000 113 | 4.900000 89.000000 114 | 4.417000 79.000000 115 | 1.700000 59.000000 116 | 4.633000 81.000000 117 | 2.317000 50.000000 118 | 4.600000 85.000000 119 | 1.817000 59.000000 120 | 4.417000 87.000000 121 | 2.617000 53.000000 122 | 4.067000 69.000000 123 | 4.250000 77.000000 124 | 1.967000 56.000000 125 | 4.600000 88.000000 126 | 3.767000 81.000000 127 | 1.917000 45.000000 128 | 4.500000 82.000000 129 | 2.267000 55.000000 130 | 4.650000 90.000000 131 | 1.867000 45.000000 132 | 4.167000 83.000000 133 | 2.800000 56.000000 134 | 4.333000 89.000000 135 | 1.833000 46.000000 136 | 4.383000 82.000000 137 | 1.883000 51.000000 138 | 4.933000 86.000000 139 | 2.033000 53.000000 140 | 3.733000 79.000000 141 | 4.233000 81.000000 142 | 2.233000 60.000000 143 | 4.533000 82.000000 144 | 4.817000 77.000000 145 | 4.333000 76.000000 146 | 1.983000 59.000000 147 | 4.633000 80.000000 148 | 2.017000 49.000000 149 | 5.100000 96.000000 150 | 1.800000 53.000000 151 | 5.033000 77.000000 152 | 4.000000 77.000000 153 | 2.400000 65.000000 154 | 4.600000 81.000000 155 | 3.567000 71.000000 156 | 4.000000 70.000000 157 | 4.500000 81.000000 158 | 4.083000 93.000000 159 | 1.800000 53.000000 160 | 3.967000 89.000000 161 | 2.200000 45.000000 162 | 4.150000 86.000000 163 | 2.000000 58.000000 164 | 3.833000 78.000000 165 | 3.500000 66.000000 166 | 4.583000 76.000000 167 | 2.367000 63.000000 168 | 5.000000 88.000000 169 | 1.933000 52.000000 170 | 4.617000 93.000000 171 | 1.917000 49.000000 172 | 2.083000 57.000000 173 | 4.583000 77.000000 174 | 3.333000 68.000000 175 | 4.167000 81.000000 176 | 4.333000 81.000000 177 | 4.500000 73.000000 178 | 2.417000 50.000000 179 | 4.000000 85.000000 180 | 4.167000 74.000000 181 | 1.883000 55.000000 182 | 4.583000 77.000000 183 | 4.250000 83.000000 184 | 3.767000 83.000000 185 | 2.033000 51.000000 186 | 4.433000 78.000000 187 | 4.083000 84.000000 188 | 1.833000 46.000000 189 | 4.417000 83.000000 190 | 2.183000 55.000000 191 | 4.800000 81.000000 192 | 1.833000 57.000000 193 | 4.800000 76.000000 194 | 4.100000 84.000000 195 | 3.966000 77.000000 196 | 4.233000 81.000000 197 | 3.500000 87.000000 198 | 4.366000 77.000000 199 | 2.250000 51.000000 200 | 4.667000 78.000000 201 | 2.100000 60.000000 202 | 4.350000 82.000000 203 | 4.133000 91.000000 204 | 1.867000 53.000000 205 | 4.600000 78.000000 206 | 1.783000 46.000000 207 | 4.367000 77.000000 208 | 3.850000 84.000000 209 | 1.933000 49.000000 210 | 4.500000 83.000000 211 | 2.383000 71.000000 212 | 4.700000 80.000000 213 | 1.867000 49.000000 214 | 3.833000 75.000000 215 | 3.417000 64.000000 216 | 4.233000 76.000000 217 | 2.400000 53.000000 218 | 4.800000 94.000000 219 | 2.000000 55.000000 220 | 4.150000 76.000000 221 | 1.867000 50.000000 222 | 4.267000 82.000000 223 | 1.750000 54.000000 224 | 4.483000 75.000000 225 | 4.000000 78.000000 226 | 4.117000 79.000000 227 | 4.083000 78.000000 228 | 4.267000 78.000000 229 | 3.917000 70.000000 230 | 4.550000 79.000000 231 | 4.083000 70.000000 232 | 2.417000 54.000000 233 | 4.183000 86.000000 234 | 2.217000 50.000000 235 | 4.450000 90.000000 236 | 1.883000 54.000000 237 | 1.850000 54.000000 238 | 4.283000 77.000000 239 | 3.950000 79.000000 240 | 2.333000 64.000000 241 | 4.150000 75.000000 242 | 2.350000 47.000000 243 | 4.933000 86.000000 244 | 2.900000 63.000000 245 | 4.583000 85.000000 246 | 3.833000 82.000000 247 | 2.083000 57.000000 248 | 4.367000 82.000000 249 | 2.133000 67.000000 250 | 4.350000 74.000000 251 | 2.200000 54.000000 252 | 4.450000 83.000000 253 | 3.567000 73.000000 254 | 4.500000 73.000000 255 | 4.150000 88.000000 256 | 3.817000 80.000000 257 | 3.917000 71.000000 258 | 4.450000 83.000000 259 | 2.000000 56.000000 260 | 4.283000 79.000000 261 | 4.767000 78.000000 262 | 4.533000 84.000000 263 | 1.850000 58.000000 264 | 4.250000 83.000000 265 | 1.983000 43.000000 266 | 2.250000 60.000000 267 | 4.750000 75.000000 268 | 4.117000 81.000000 269 | 2.150000 46.000000 270 | 4.417000 90.000000 271 | 1.817000 46.000000 272 | 4.467000 74.000000 273 | -------------------------------------------------------------------------------- /data/gaussian.txt: -------------------------------------------------------------------------------- 1 | 7.831273165323071606e-01 3.311374331493355805e-01 2 | -8.034393864714645117e-02 1.202349231474020508e+00 3 | -5.830228785232781652e-01 -1.094918463527496888e+00 4 | -1.438168566084212285e+00 -1.099510169965831619e+00 5 | 4.965026559111671012e-01 -4.845601413039231642e-01 6 | -8.539810950692980906e-01 8.183564185334292473e-02 7 | -3.872943192132554443e-03 -1.301533566535465702e+00 8 | -1.384778641404214028e+00 1.395508223002047687e+00 9 | 2.672747271651349510e-01 -1.284860728050021272e+00 10 | -1.957368273439064721e+00 -1.154585569364284742e-03 11 | 4.213838947864212425e-01 7.635789750057514480e-01 12 | -1.726476414864609010e+00 -1.462326154470155126e+00 13 | 1.354492978443073481e+00 -3.196381587659788970e-01 14 | 1.989100351595446126e-01 -1.351795816975342790e+00 15 | -4.117907487287491564e-01 -1.577435190625841477e+00 16 | 5.179470893750463822e-02 7.764578265920937250e-01 17 | 1.391216682519869741e+00 1.225048848623052189e+00 18 | -1.960343465054615653e+00 8.320805108034556863e-01 19 | -2.237439709316521863e-01 -2.009509692089570088e+00 20 | -2.193303004351861846e-02 9.040178943577144022e-01 21 | -6.836303821922724566e-01 7.477272790061201269e-01 22 | -5.925949444997211102e-02 1.520431288225711830e+00 23 | 6.170656124607146742e-01 3.259861638869966383e-01 24 | 1.707544218367208211e-01 -1.251733539624940494e+00 25 | 3.476621980569136738e-01 7.520756098926940636e-01 26 | 6.440750921644607940e-01 -1.625727548110448284e+00 27 | -5.147803823375869392e-01 4.388634569744080949e-01 28 | 5.276205563936936516e-01 9.210181561256376437e-01 29 | 5.140424165316244132e-01 -4.350534142541132715e-02 30 | -7.671851124888534601e-01 -1.961016738049034425e+00 31 | 1.826800365485481414e+00 6.307653280730297629e-01 32 | 3.558567815664661071e-01 -9.522494822680839244e-01 33 | 1.025865743701772637e+00 -8.965674570816559719e-01 34 | -1.458407414813187586e+00 -7.714798318372746433e-01 35 | -4.347316187684101274e-02 2.828046235276553966e-01 36 | 1.583510619214418602e+00 2.586594191236604168e-01 37 | -1.224793117887762683e+00 -3.216057784276787901e-01 38 | 1.444988572525659709e+00 6.208319882541666335e-01 39 | -1.417237125931250619e+00 -3.544104762089057142e-02 40 | 1.065322643171376094e-01 -3.068568891404723775e-01 41 | -1.175143711627312682e+00 -8.055245337066987421e-01 42 | 1.240606463188312691e+00 6.406922491774866835e-01 43 | 1.106631540451916473e+00 3.359679533478458491e-01 44 | 1.476487681081460002e+00 -4.813681855087658112e-01 45 | -1.159035770757212758e+00 -2.046820643747513102e-01 46 | -3.864115768104467441e-01 5.463352893092536799e-01 47 | 3.673219044709144510e-01 -5.756417648655788311e-01 48 | -6.080033330367284883e-01 -1.430706152406878706e+00 49 | -2.010632843971063455e+00 -6.362591800892722915e-01 50 | 6.648365173664215755e-01 4.588789662690598292e-01 51 | 1.028624359081929951e+00 4.917575131680584910e-01 52 | 5.512855942023776878e-02 2.980489081135884177e-01 53 | 6.078327259421067996e-01 6.634041807511674715e-01 54 | 1.405561099156142635e-01 4.491760741400366852e-01 55 | -8.288736298536243385e-01 -1.539116839839498596e-01 56 | -4.689150147619550735e-01 -1.418294345796908829e-01 57 | -8.883760141916452469e-01 -3.757625783136466224e-01 58 | -8.149792886071504094e-01 -4.071932563200470478e-01 59 | 1.666644676468339026e-01 -1.127215631680740460e+00 60 | 1.208076119755006894e-01 2.192303451914634771e-01 61 | 1.212990772218605517e+00 -1.052125317822423023e+00 62 | 3.850830204694908332e-01 1.653609887071909812e+00 63 | -5.543686234832529980e-01 -6.644184549504019710e-01 64 | -1.631488825445948310e+00 -1.876350849130509424e+00 65 | 7.962457539194717215e-01 -6.287830820066309112e-01 66 | -4.230699913896637798e-01 9.949429673943239738e-01 67 | 1.678272271926155024e-01 7.160448148429473525e-01 68 | -9.082690323354868367e-01 -6.737238849307058963e-01 69 | 2.330570190776818651e-01 -1.918761888098308832e+00 70 | -9.789048359655517784e-01 -2.639395573401190020e+00 71 | -1.115297029099354686e+00 8.624016376750763824e-01 72 | -1.087280876234507421e+00 -8.404320091592398789e-01 73 | -5.153425381007507022e-01 1.210947868310442876e+00 74 | 1.286882892943597811e+00 -9.164616091325827885e-01 75 | -1.167960981362395012e+00 -3.142153879186990184e-01 76 | 1.683861780819099652e+00 2.157634203833092290e-01 77 | 1.203762980917654435e+00 2.364104177624835934e+00 78 | -4.417892327798630947e-01 8.425789673151053982e-01 79 | 5.885841295357083780e-01 6.402214915112304505e-02 80 | 6.819238423288550566e-01 2.294421014318900109e+00 81 | 1.468660876108261126e+00 -3.045977832734047749e-02 82 | 1.201397150587107276e+00 8.044950575668279313e-01 83 | -3.897832665435173038e-01 9.297396208296869746e-01 84 | -2.000255925535797008e+00 -3.171236068792100404e-01 85 | -2.281668494383551904e-01 -3.652962667448894640e-01 86 | -2.369792703832634961e-02 -1.673465466706403504e+00 87 | -8.078664681436773520e-01 -6.514567079181162734e-01 88 | -1.499865492237151932e+00 -8.157725714146476514e-01 89 | -1.407640696933772473e+00 8.970289486823135316e-01 90 | 6.978639791360048417e-01 -2.598384479378265666e-01 91 | -4.158768960857405217e-01 -7.952136698895314604e-01 92 | -2.900734091708069284e-01 6.637165326746511385e-01 93 | -1.024504332504579729e+00 -1.010470609422004173e+00 94 | -1.098809969426923328e+00 -1.088462239639364504e-01 95 | -1.105967282644725325e+00 8.695123954781207587e-01 96 | -2.963092464308123009e-01 7.931876998731864914e-01 97 | 2.088098605838754618e-01 7.358318672170756081e-01 98 | -1.188585784129907008e+00 1.491330771631730556e+00 99 | -8.852680531703300826e-01 1.434887001318051025e+00 100 | -1.095802014417969428e+00 7.551635838233029530e-01 101 | 4.723956114410309048e+00 -3.097032141841571384e+00 102 | 2.400843031756599633e+00 -2.310525750627216990e+00 103 | 2.625310418275604185e+00 -1.498716665727555242e+00 104 | 3.175425637441918525e+00 -3.085534326072108868e+00 105 | 3.023481515548407383e+00 -1.865298374864685194e+00 106 | 4.022485547563077191e+00 -2.646738644458051404e+00 107 | 2.878526899158495134e+00 -3.635691826856400688e+00 108 | 5.729465423677392977e+00 -8.020196829195902488e-01 109 | 2.211665906255475100e+00 -3.323199119990741579e+00 110 | 1.664765997066131931e+00 -3.318616804788903973e+00 111 | 3.135771899448683619e+00 -2.587945996584591235e+00 112 | 3.376815667347749184e+00 -2.866645697540397997e+00 113 | 2.421002946499495323e+00 -4.143585810500879596e+00 114 | 3.848899226311016442e+00 -1.739106455487308134e+00 115 | 4.578696287061083225e+00 -1.921638152671235389e+00 116 | 1.777740060884328788e+00 -4.222127820409778742e+00 117 | 2.776310486115989740e+00 -3.595724386132248540e+00 118 | 3.428745229353554080e+00 -2.692675525849730356e+00 119 | 4.241367893814862455e+00 -7.035742299156759749e-01 120 | 2.362102468155359958e+00 -3.246404794997480803e+00 121 | 3.762488518494844669e+00 -2.940250791063682811e+00 122 | 9.265999998270419979e-01 -3.252435647867608637e+00 123 | 2.913945279878516370e+00 -4.485388149020538506e+00 124 | 1.944521526821221258e+00 -3.370064116617669026e+00 125 | 3.390523398126874532e+00 -2.070144990945720576e+00 126 | 3.509041300164173371e+00 -3.361491536589907625e+00 127 | 3.887654567172009301e+00 -2.445588218311487871e+00 128 | 1.759399016011158334e+00 -4.668144713902065313e+00 129 | 2.820520923622204634e+00 -2.483841143925478612e+00 130 | 2.657970349546819211e+00 -2.313440994814314866e+00 131 | 2.128898421149386433e+00 -3.550703219648246645e+00 132 | 1.212774926095703254e+00 -4.084531510523314424e+00 133 | 1.449165257981847033e+00 -3.444062496424010078e+00 134 | 2.977820256291614509e+00 -3.891511976686772734e+00 135 | 3.461907537947769065e+00 -3.703271628843427976e+00 136 | 3.113000823290659191e+00 -3.834908339075092432e+00 137 | 1.391209494770654143e+00 -3.497836633775200710e+00 138 | 3.927466516944022601e-02 -3.800512383369254898e+00 139 | 1.824743596905845866e+00 -2.727451180907364492e+00 140 | 3.390137858325279474e+00 -2.068492682622482981e+00 141 | 4.363102126392604418e+00 -3.499311945529845147e+00 142 | 6.123877891045284017e-01 -4.000005505709346210e+00 143 | 3.198411420367961444e+00 -2.881607387444109492e+00 144 | 2.517193530187189943e+00 -4.981213986076811651e+00 145 | 3.588288374268815861e+00 -2.089225809373859821e+00 146 | 2.939268631344861049e+00 -2.387381193217688935e+00 147 | 4.117692274898983484e+00 -4.587293119763331539e-01 148 | 5.679220927258058715e+00 -1.870141535513561326e+00 149 | 3.662263948629511656e+00 -3.128006863511191504e+00 150 | 3.367611059724308742e+00 -2.986176148743337588e+00 151 | 2.887682145672180845e+00 -3.655066580981260582e+00 152 | 1.722729464868673066e+00 -4.807364472356918661e+00 153 | 2.605362114365814108e+00 -4.845386972291107597e+00 154 | 2.077926972831854791e+00 -3.726685730702642640e+00 155 | 3.410147901910731694e+00 -1.989468811816415883e+00 156 | 2.525171072835143082e+00 -3.879014783652046994e+00 157 | 4.559740712063875989e+00 -1.921962520997942070e+00 158 | 4.236416405111933514e+00 -1.471144335463014263e+00 159 | 2.701340393055287947e+00 -3.254422863671359689e+00 160 | 4.963622749318457927e+00 -2.946014099395848795e+00 161 | 2.663633789821224784e+00 -3.387348746225876450e+00 162 | 2.801888556836968469e+00 -4.281665345529771294e+00 163 | 3.573479009323414779e+00 -2.576976153267277247e+00 164 | 4.424847194843827936e+00 -1.291926224478855856e+00 165 | 2.981193407190906086e+00 -4.310839693975630382e+00 166 | 1.751186183189368695e+00 -4.135634060697300107e+00 167 | 2.573390116742911182e+00 -3.540519957505841475e+00 168 | 3.112410191023293304e+00 -2.275608447968706649e+00 169 | 1.659763855190864668e+00 -4.502247628332449736e+00 170 | 3.396597051980000614e+00 -2.594945335740749037e+00 171 | 3.750495951098211656e+00 -4.665628213044219486e-01 172 | 4.938041259372932146e+00 -7.094999915768900323e-01 173 | 2.649530567125427005e+00 -2.962462109017429679e+00 174 | 2.607019960425190241e+00 -2.157891341870037394e+00 175 | 3.284150209026160727e+00 -3.041034581989959573e+00 176 | 3.831939228220544003e+00 -2.758875101300976240e+00 177 | 2.967653413600410239e+00 -2.401738158858948147e+00 178 | 1.906007422174770483e+00 -3.431689183146363398e+00 179 | 4.589428444769229287e+00 -2.589888262217007409e+00 180 | 2.903368999895721281e+00 -3.655525624377446992e+00 181 | 2.197754634468755519e+00 -4.833701123411407785e+00 182 | 3.607270511143777725e+00 -1.214794243639380822e+00 183 | 2.980098766987289594e+00 -3.332448572808980547e+00 184 | 2.663176471629321806e+00 -3.031559366997206961e+00 185 | 2.626290935206070376e+00 -2.596808274466897437e+00 186 | 3.766946485176024595e+00 -2.103713800006061785e+00 187 | 4.550248652643283265e+00 -2.879197257092712903e+00 188 | 2.492674186416972137e+00 -3.886160192179664019e+00 189 | 2.905755176705449028e+00 -2.067704566319920723e+00 190 | 1.485989367620679458e+00 -4.420974549030029621e+00 191 | 3.880833450733935930e+00 -2.085952637687524813e+00 192 | 3.823161434310542006e+00 -2.921596908333589759e+00 193 | 4.201448129435638279e+00 -3.741663813955418583e+00 194 | 4.271561152991329990e+00 -2.423385287044157455e+00 195 | 1.692639280782996636e+00 -4.164975213037132029e+00 196 | 2.886481706890924315e+00 -2.575569541309370880e+00 197 | 3.557697765352831443e+00 -3.199190046166999757e+00 198 | 3.929044983109027278e+00 -2.091746604749135319e+00 199 | 2.327650246099407294e+00 -3.036898369026346689e+00 200 | 3.920138220332397783e+00 -3.128566058031765351e+00 201 | 3.641122759724378266e+00 3.271937577004622177e+00 202 | 4.223343228484539580e+00 2.856182365300890424e+00 203 | 4.661391735755238130e-01 3.958473216372399239e+00 204 | 4.195877484800403678e+00 3.559908764978545026e+00 205 | 1.251417676696441106e+00 3.836783950798944076e+00 206 | 5.205670619632452478e+00 2.107181859341590080e+00 207 | 3.408730474179610503e+00 3.935404843396899111e+00 208 | 3.333957727602621546e+00 2.666545432105547064e+00 209 | 2.788618277304233395e+00 3.306826554964377163e+00 210 | 3.615976643642668353e+00 2.738703607596255196e+00 211 | 1.988202705404192194e+00 3.370784099462037720e+00 212 | 2.741654062241571932e+00 3.700018784684822926e+00 213 | 1.515820373292507295e+00 2.537803812077046750e+00 214 | 1.724294911513367445e+00 3.675212050961635502e+00 215 | 4.849080480752861355e+00 2.622908621320917621e+00 216 | 3.576008864472093762e+00 3.364212687924691636e+00 217 | 3.595976375890078458e+00 1.589820141251748531e+00 218 | 4.394909264654188519e+00 2.711954210415099986e+00 219 | 2.395030977999411448e+00 3.332862305639274325e+00 220 | 4.005242052933107644e+00 4.435281053496727743e+00 221 | 2.787127080054889294e+00 4.490004935645016459e+00 222 | 2.847290544727122619e+00 3.320027956348200249e+00 223 | 1.358074358611387478e+00 4.472686702314321749e+00 224 | 3.794274828094430063e+00 3.250970520393531960e+00 225 | 3.250774007720355296e+00 2.378428342808077911e+00 226 | 4.028696704860527333e+00 3.128266309453969107e+00 227 | 3.495745984406119611e+00 9.777964440398982937e-01 228 | 4.156311884369554122e+00 2.610274537000245143e+00 229 | 3.938112061047914114e+00 2.507087843574419050e+00 230 | 1.380152071375863265e+00 2.419472810382238048e+00 231 | 1.652445656405174823e+00 3.763554555116671363e+00 232 | 2.101124396945837081e+00 3.456053623022113719e+00 233 | 2.312852031750637138e+00 3.836404541576776683e+00 234 | 5.511322074585741859e+00 1.956261174009414683e+00 235 | 4.457176556207950391e+00 7.814796664523333547e-01 236 | 3.810573151154033500e+00 2.648103094910281730e+00 237 | 3.572509216427552037e+00 3.836617120784866053e+00 238 | 3.584298148030899522e+00 1.602114826171295192e+00 239 | 1.579671088035696336e+00 3.426006225672014960e+00 240 | 3.205011900176368833e+00 3.522780880292504690e+00 241 | 2.675307138359487080e+00 4.129578135821844853e+00 242 | 1.741837391776150046e+00 4.979552683804588042e+00 243 | 5.114355901534750615e+00 9.313696566165474700e-01 244 | 2.650038773281031546e+00 2.866271644134991359e+00 245 | 3.341098543034932700e+00 4.191580642628142783e+00 246 | 3.649404543034684867e+00 3.580733023900394940e+00 247 | 1.935073942224376564e+00 1.820513705061402643e+00 248 | 5.175749099446971258e+00 1.686299284737738047e+00 249 | 3.018680405761722252e+00 2.579439737058648596e+00 250 | 2.577277527633886400e+00 2.654439971705995216e+00 251 | 2.090240162962172299e+00 3.672434514488726709e+00 252 | 2.840081811208595486e+00 3.152876375994440927e+00 253 | 3.264493514626605641e+00 2.497807459034977029e+00 254 | 2.884843113212905763e+00 2.767582566041744130e+00 255 | 3.913424478728149580e+00 2.820211288231186497e+00 256 | 2.184116243842652683e+00 1.982775892332458767e+00 257 | 3.504261821962962653e+00 1.591799557809988608e+00 258 | 2.031457074201070689e+00 3.493759489642203953e+00 259 | 2.867403210375026212e+00 2.896622476943235380e+00 260 | 2.180304991911550250e+00 2.488616911299223133e+00 261 | 3.251751426901100750e+00 1.816415904529030900e+00 262 | 5.215510613453311350e+00 2.561682863412608313e+00 263 | 2.832633449630286293e+00 4.802140255131490676e+00 264 | 4.071507386993500788e+00 2.309941924143174230e+00 265 | 3.845308823406377918e+00 1.623395824781930674e+00 266 | 2.157596969296755152e+00 1.602181425130183356e+00 267 | 2.956247738217866683e+00 2.707800084739138136e+00 268 | 2.407129865833626425e+00 4.658182554319839852e+00 269 | 3.707250467218472512e+00 2.690929296876084287e+00 270 | 3.457382430251321193e+00 2.887962198494876898e+00 271 | 3.888466464961328306e+00 1.260471428915036496e+00 272 | 2.083921976160045819e+00 4.102649016869759357e+00 273 | 5.512263816481621781e-01 4.281009419650771974e+00 274 | 4.285880747827721216e+00 2.169246969347888410e+00 275 | 1.089680943892389342e+00 2.861107425257179315e+00 276 | 2.927144420488598797e+00 2.375845376794357477e+00 277 | 3.509967858883638936e+00 2.451315029303243076e+00 278 | 2.890953105322251027e+00 3.319786687444026274e+00 279 | 3.182513330806591600e+00 4.612987674578398867e+00 280 | 2.496580456484981170e+00 4.150442055860635193e+00 281 | 2.617639865710220004e+00 2.081755303365161680e+00 282 | 1.553569196905900451e+00 4.298818855496965519e+00 283 | 3.175176550678842524e+00 2.913134449309134677e+00 284 | 3.714659176058908496e+00 2.249782198563925917e+00 285 | 3.542315113863680143e+00 2.711854078800752355e+00 286 | 3.755549439670429201e+00 6.728577159206650116e-01 287 | 9.636809205928278388e-01 4.171173442161575373e+00 288 | 6.626073379043766920e+00 2.178098343571706064e+00 289 | 3.110485384828256095e+00 3.264277461374361700e+00 290 | 4.763231526787874870e+00 1.519527720638305324e+00 291 | 3.473157375240658595e+00 4.256932648150556275e+00 292 | 2.223166900427777204e+00 3.657160179529912902e+00 293 | 4.339484380929386376e+00 2.261538757466381888e+00 294 | 4.094740981373528044e+00 2.671506838296993980e+00 295 | 3.516769909600331534e+00 3.359978483342690669e+00 296 | 2.346109892407458375e+00 3.218217276618055234e+00 297 | 1.464753080968669519e+00 3.322032039172307716e+00 298 | 2.770396618578629511e+00 3.154902723960185007e+00 299 | 2.066965263094424632e+00 2.446067754367644564e+00 300 | 4.722638879302212267e+00 1.878556174202429485e+00 301 | -5.227573416162130115e+00 2.660861128005476139e-01 302 | -2.396993643281510877e+00 3.122983021137724968e+00 303 | -2.830501867448424136e+00 3.109262449658256600e+00 304 | -2.975823350158717684e+00 3.879899706678513915e+00 305 | -2.223844214132174812e+00 4.362289309947494864e+00 306 | -5.870142048245820199e+00 5.081644488908905188e-01 307 | -1.659814217903492040e+00 3.823795430259286476e+00 308 | -1.821252688288993449e+00 2.785222388930558512e+00 309 | -2.418141825507006182e+00 3.043739484577745458e+00 310 | -2.948526874515569052e+00 4.111895379372638715e+00 311 | -3.734804006575842550e+00 3.412048463836103274e+00 312 | -4.218209463515610658e+00 2.228163418110260086e+00 313 | -4.758606165729551307e+00 1.355896930105059184e+00 314 | -2.301208620003576399e+00 2.700251151745801881e+00 315 | -3.062149100291124526e+00 3.072504627420380441e+00 316 | -4.866651398390477112e+00 8.381982939517351738e-01 317 | -1.274470574338894213e+00 4.649876359858349240e+00 318 | -2.993046049316300916e+00 3.478860970107038764e+00 319 | -3.140694727361936778e+00 2.529665786466437361e+00 320 | -2.835377064982459672e+00 2.924805777808102025e+00 321 | -2.607687553721986351e+00 2.691633776802734523e+00 322 | -1.812634943648342656e+00 4.184026902087854438e+00 323 | -2.779007482115104288e+00 1.245831811894971386e+00 324 | -3.441490038329656276e+00 4.133748572439177593e+00 325 | -2.203919824513231873e+00 2.139907850868583150e+00 326 | -1.972391307111230452e+00 3.986434872965681553e+00 327 | -2.172878461034731146e+00 2.553459937703768112e+00 328 | -2.604165260630713963e+00 3.322194886688950355e+00 329 | -3.556286593121673612e+00 5.192574134168269850e+00 330 | -3.565530039032245302e+00 3.265134346383410158e+00 331 | -4.138509458237818350e+00 1.069388460830587029e+00 332 | -2.377759739922548299e+00 3.912823251047493400e+00 333 | -1.282496013992534767e+00 2.725165902476136548e+00 334 | -2.956853261115960141e+00 3.772364185807445480e+00 335 | -4.063670990719186804e+00 2.426699737843126758e+00 336 | -3.195619372321504059e+00 2.881196978146739696e+00 337 | -4.319123816502175472e+00 4.156237423638252437e+00 338 | -3.684660293597155523e+00 2.558997615954101246e+00 339 | -1.626276035302672618e+00 3.916546997700764088e+00 340 | -3.723927130119000228e+00 1.888288537255559785e+00 341 | -2.111449152920156003e+00 3.388810026958788857e+00 342 | -9.427474560914252955e-01 4.287996214217316471e+00 343 | -1.564383997070189380e+00 2.687541289368067687e+00 344 | -1.750881379665986115e+00 4.592480744398164560e+00 345 | -9.489355951339497608e-01 3.444363890998247690e+00 346 | -3.609825853840869847e+00 3.388201269687353800e+00 347 | -1.857901367767549772e+00 2.931881974024800552e+00 348 | -1.745633121940045385e+00 3.174437065794283974e+00 349 | -1.734962803253844443e+00 2.014748385339985237e+00 350 | -2.976939088989435156e+00 3.586438309065362429e+00 351 | -3.427738854492742959e+00 3.428090399472771388e+00 352 | -4.659630795134283687e+00 2.881219232570361832e+00 353 | -3.241176378257666801e+00 3.469788156870182938e+00 354 | -3.966840712244460221e+00 3.347264821895395759e+00 355 | -4.171279851569346775e+00 2.041764060830162819e+00 356 | -1.362557576651595603e+00 2.551804801737802109e+00 357 | -3.895760913791549207e+00 2.044057259250594338e+00 358 | -2.136241485926029693e+00 2.757370971882769251e+00 359 | -3.206627755259880086e+00 3.716875219855354295e+00 360 | -3.626747169147763294e+00 3.321956181346450432e+00 361 | -3.606899268094227118e+00 2.715421852772396782e+00 362 | -3.730085370879788531e+00 9.062747959744843484e-01 363 | -3.473410993476941933e+00 2.053468179842293573e+00 364 | -3.355797292290547240e+00 3.711875285091906562e+00 365 | -3.515602103074075835e+00 3.221492732275266668e+00 366 | -2.978801558216458201e+00 2.650034794270530458e+00 367 | -4.603254865810645136e+00 3.000408694879287630e+00 368 | -2.363851324212862259e+00 3.499001794014055200e+00 369 | -3.014622745922354685e+00 3.532560738575713621e+00 370 | -3.304446588404025498e+00 3.779502245318445919e+00 371 | -2.401875446007320392e+00 2.661039942921080925e+00 372 | -2.862349446767081140e+00 4.051865643790758753e+00 373 | -2.487555372410748866e+00 3.591266555264122751e+00 374 | -1.495042349964259332e+00 4.217293410842253287e+00 375 | -1.614195457093534403e+00 4.722927807148923129e+00 376 | -3.407960229310635913e+00 2.877118684034060792e+00 377 | -1.155751210670381557e+00 3.864858372416128063e+00 378 | -1.948739341506597444e+00 4.562976122115752098e+00 379 | -2.471346835021172694e+00 3.217860167260357507e+00 380 | -2.059899075269430835e+00 6.118268256104551384e+00 381 | -4.236649012931037461e+00 2.061082869311441357e+00 382 | -2.908747233067384474e+00 3.117856327024731922e+00 383 | -2.670374893329237942e+00 2.161413694163270005e+00 384 | -1.858085295798081127e+00 3.405478595241451245e+00 385 | -2.615716590107928230e+00 3.539880716723161846e+00 386 | -3.512576386412799234e+00 2.171340121561629033e+00 387 | -2.265983367966339035e+00 2.778496473675498901e+00 388 | -2.223748572245476840e+00 5.280464922497163371e+00 389 | -3.860658613056298094e+00 2.527335582687912474e+00 390 | -3.379829325943274121e+00 3.025423250133788322e+00 391 | -3.808383857456959287e+00 2.197445616200791818e+00 392 | -2.906383201905091251e+00 3.538453873685972528e+00 393 | -3.814958931272343090e+00 3.577644578520760454e+00 394 | -3.394624543319981314e+00 1.238386880402086332e+00 395 | -2.334032970219134029e+00 2.280926516793696734e+00 396 | -4.418040306283968377e+00 1.825925936351721512e+00 397 | -3.082665835855546099e+00 3.124088149377696855e+00 398 | -6.606334041901384957e-01 5.178105207481491945e+00 399 | -3.939213681240249620e+00 2.176483314422138271e+00 400 | -2.155721273780297764e+00 2.244017520381464958e+00 401 | -3.639372167763556654e+00 -2.215012700913883137e+00 402 | -3.863825396549420255e+00 -1.753166743573073560e+00 403 | -4.821004960554998497e+00 -1.411722723112133648e+00 404 | -3.528346740583587771e+00 -2.607631534794863537e+00 405 | -1.806049134622278673e+00 -4.158631594232883977e+00 406 | -3.877142085750687528e+00 -3.353481793493349628e+00 407 | -2.486565756720902076e+00 -4.137379162439597913e+00 408 | -2.383017595462230798e+00 -3.900196594158484409e+00 409 | -2.650251197199670283e+00 -2.932647114233481123e+00 410 | -3.553872075554419752e+00 -2.089179178805802284e+00 411 | -4.570109377857159316e+00 -3.026707897919325596e+00 412 | -1.899278302107419991e+00 -3.420904747219386088e+00 413 | -2.067755466401890896e+00 -2.328105329262957657e+00 414 | -2.601020052792607729e+00 -2.108306063704452615e+00 415 | -2.973381326487933407e+00 -2.654148665551427477e+00 416 | -3.871475661596618600e+00 -3.031254133435508535e+00 417 | -2.986079539582028097e+00 -1.638926758724983612e+00 418 | -3.935780778223058718e+00 -3.107367151410684425e+00 419 | -5.217301091842369232e+00 -2.670105384291582951e+00 420 | -4.156981569129702514e+00 -3.285708762261072113e+00 421 | -2.620481947054313476e+00 -3.339155516082428665e+00 422 | -3.175164540052376694e+00 -2.148709829830187878e+00 423 | -3.154395424171087203e+00 -2.494190220561245841e+00 424 | -2.146838933755926604e+00 -2.514513400347815786e+00 425 | -3.231359502457025634e+00 -2.651966716973804594e+00 426 | -3.137613111554782730e+00 -2.521647825467639326e+00 427 | -4.219849113597000212e+00 -2.508812804641034511e+00 428 | -1.098674791204454770e+00 -3.382370029035823400e+00 429 | -1.458899983072620810e+00 -3.830182372066480934e+00 430 | -3.620866126084155034e+00 -6.946616022980984795e-01 431 | -4.172266736680315447e+00 -1.555210978551449852e+00 432 | -2.859618644838113788e+00 -1.580862528115815291e+00 433 | -3.551506975360436691e+00 -1.854145867697488237e+00 434 | -3.443763516609644970e+00 -3.978332799803528275e+00 435 | -2.746238718538846157e+00 -1.835270713205112703e+00 436 | -1.997708861177798889e+00 -3.802837222037275922e+00 437 | -1.667062258973185163e+00 -5.459791957970075771e+00 438 | -3.036618484017117758e+00 -3.653134129337655356e+00 439 | -3.202373467115486871e+00 -2.849466483693825936e+00 440 | -2.271056168791053764e+00 -3.962370616529667267e+00 441 | -3.221439923625369683e+00 -3.196269772264231701e+00 442 | -3.294847391838253259e+00 -2.619411958822395192e-01 443 | -1.014839929677366248e+00 -5.262949764509423645e+00 444 | -2.531226375235309689e+00 -1.659718072575526726e+00 445 | -4.085366031427200895e+00 -1.343527467209188142e+00 446 | -2.384742214302148611e+00 -3.482917437026740615e+00 447 | -1.192807443284511049e+00 -4.773126477662290590e+00 448 | -3.358004913202710551e+00 -2.826013025938583834e+00 449 | -4.042784534889972470e+00 -2.412414718958307169e+00 450 | -2.485382937098089773e+00 -2.110693132345399459e+00 451 | -2.802312794085224823e+00 -9.681285609720249319e-01 452 | -2.067010886469461628e+00 -4.438347970655525998e+00 453 | -3.805749537052184550e+00 -3.150589205063367704e+00 454 | -2.638195132849252555e+00 -1.935392967307742573e+00 455 | -2.669370120266342461e+00 -2.603701773956946575e+00 456 | -3.399087062323681074e+00 -3.013662667350774704e+00 457 | -3.796102119946848319e+00 -1.609402567079751822e+00 458 | -2.744598773827945148e+00 -4.054459859352970597e+00 459 | -2.527751940502791950e+00 -3.102552369315247915e+00 460 | -4.093475801339799602e+00 -1.884125268170766754e+00 461 | -3.293271394248842299e+00 -3.925977348271404654e+00 462 | -2.903927049445427233e+00 -2.876710345319445139e+00 463 | -2.967542589504066797e+00 -1.827732156010007492e+00 464 | -2.851527155436179406e+00 -2.428458297002959121e+00 465 | -1.891729304448361848e+00 -3.078274064513154773e+00 466 | -2.414430321253924205e+00 -4.165434319277974140e+00 467 | -2.690141397536769841e+00 -4.448998463452866758e+00 468 | -3.591973236324695229e+00 -3.368063580643054777e+00 469 | -4.981919664223966926e+00 -2.566365238504477286e+00 470 | -2.706666479131010217e+00 -2.476632265030114244e+00 471 | -2.312926780870921739e+00 -2.367809817920232263e+00 472 | -3.831576668195582513e+00 -3.705244120503393024e+00 473 | -8.300191156767020573e-01 -5.643743536752600320e+00 474 | -1.327439311764874397e+00 -3.898270547143378728e+00 475 | -3.911943449072431633e+00 -2.906946247502488756e+00 476 | -4.271882626665416183e+00 -2.237035573552429213e+00 477 | -3.012561933763264754e+00 -2.164360934640446210e+00 478 | -3.074037547115485491e+00 -2.640663828742737085e+00 479 | -4.090338842501643590e+00 -2.285721961172527905e+00 480 | -3.263496702725670495e+00 -2.042400193739325687e+00 481 | -2.795720352116779051e+00 -2.296440528358294486e+00 482 | -4.049083121735790769e+00 -1.335848747089964927e+00 483 | -4.916449553286952323e+00 -3.501825479566200272e+00 484 | -1.899826318598367703e+00 -2.894053003000305768e+00 485 | -3.081031668501691190e+00 -2.692354362930844403e+00 486 | -4.328826902156395207e+00 -3.743068840553721355e+00 487 | -3.490681775882753968e+00 -3.387238581020456163e+00 488 | -2.646286033614841493e+00 -2.243429567338732333e+00 489 | -4.148840707913274528e+00 -4.551253580889238037e+00 490 | -4.710534596735468149e+00 -1.392241492989722662e+00 491 | -5.386915188287884071e+00 -2.453759667000752209e+00 492 | -2.305121089540011603e+00 -4.593684118371613501e+00 493 | -1.828978752772605354e+00 -3.084911234030334182e+00 494 | -3.434911354974753195e+00 -3.897440112687716507e+00 495 | -2.319161217778244577e+00 -2.213090923343787875e+00 496 | -2.029825067663789007e+00 -3.708680661247641730e+00 497 | -2.124151344443151235e+00 -3.510671289576799303e+00 498 | -2.128369498711510932e+00 -2.420614685641592878e+00 499 | -1.722665624935804107e+00 -3.160469555628295346e+00 500 | -4.235101433917340508e+00 -1.593653525680852168e+00 501 | -------------------------------------------------------------------------------- /data/img/graphical_models.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/juliennonin/variational-gaussian-mixture/3785528a5dfe63fc879ebfdae7df0e48978cf058/data/img/graphical_models.png -------------------------------------------------------------------------------- /data/img/results_fullGMM_OF.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/juliennonin/variational-gaussian-mixture/3785528a5dfe63fc879ebfdae7df0e48978cf058/data/img/results_fullGMM_OF.png -------------------------------------------------------------------------------- /demo.py: -------------------------------------------------------------------------------- 1 | #%% 2 | %load_ext autoreload 3 | %autoreload 2 4 | 5 | import numpy as np 6 | import matplotlib.pyplot as plt 7 | import src as mixture 8 | 9 | 10 | #%% Synthetic 1 11 | m_true = np.array([[0, 0], [3, -3], [3, 3], [-3, 3], [-3, -3]]) 12 | covs_true = np.array([[[1, 0], [0, 1]], [[1, 0.5], [0.5, 1]], [[1, -0.5], [-0.5, 1]], [[1, 0.5], [0.5, 1]], [[1, -0.5], [-0.5, 1]]]) 13 | X = np.concatenate([np.random.multivariate_normal(m_true[k], covs_true[k], 100) for k in range(len(m_true))]) 14 | 15 | #%% Synthetic 2 16 | m_true = np.array([[0, -2],[0, 0], [0, 2]]) 17 | covs_true = np.array([[[2, 0], [0, 0.2]], [[2, 0], [0, 0.2]], [[2, 0], [0, 0.2]]]) 18 | X = np.concatenate([np.random.multivariate_normal(m_true[k], covs_true[k], 100) for k in range(len(m_true))]) 19 | 20 | #%% Synthetic 3 21 | m_true = np.array([[0, 0], [0, 0], [0, 0]]) 22 | covs_true = np.array([[[1, 0], [0, 0.2]], [[0.02, -0.08], [-0.08, 1.5]], [[0.5, 0.4], [0.4, 0.5]]]) 23 | X = np.concatenate([np.random.multivariate_normal(m_true[k], covs_true[k], 100) for k in range(len(m_true))]) 24 | 25 | #%% Olf Faithful data set 26 | X = np.loadtxt('data/faithful.txt') 27 | 28 | 29 | #%% 30 | # Standardize the data 31 | X = (X - X.mean(axis=0)) / X.std(axis=0) 32 | 33 | # model = VariationalGaussianMixture(K=10, display=True, max_iter=200, plot_period=5, init_param="kmeans") 34 | model = mixture.VariationalGaussianMixtureCB(K=10, display=True, max_iter=201, plot_period=20, init_param="kmeans") 35 | model.fit(X) 36 | 37 | #%% Display ELBO 38 | plt.plot(model.elbo, 'k') 39 | plt.margins(x=0) 40 | plt.xlabel('iteration') 41 | plt.ylabel('Variational lower bound') 42 | plt.show() 43 | 44 | plt.plot(model.elbo[:-1] - model.elbo[1:], 'k') 45 | plt.margins(x=0) 46 | plt.xlabel('iteration') 47 | plt.ylabel('Variation of the Variational lower bound') 48 | plt.axhline(y=0) 49 | plt.show() 50 | 51 | if not all((model.elbo[:-1] - model.elbo[1:]) <= 1e-7): 52 | raise ValueError("Implementation Error: ELBO should be increasing") 53 | #%% Display final GM 54 | model._display_2D(X, n_levels=21, show_components=True) 55 | plt.show() 56 | 57 | 58 | # %% 59 | -------------------------------------------------------------------------------- /src/__init__.py: -------------------------------------------------------------------------------- 1 | from .viGMM_CB import VariationalGaussianMixtureCB 2 | from .viGMM_full import VariationalGaussianMixture -------------------------------------------------------------------------------- /src/base.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import matplotlib.pyplot as plt 3 | from sklearn.cluster import KMeans 4 | from utils import plot_confidence_ellipse 5 | from scipy.stats import multivariate_normal 6 | from scipy.special import digamma, gammaln, logsumexp 7 | from matplotlib.colors import LogNorm 8 | from abc import abstractmethod 9 | 10 | def log_wishart_B(invW, nu): 11 | D = len(invW) 12 | return + 0.5 * nu * np.log(np.linalg.det(invW)) \ 13 | - 0.5 * nu * D * np.log(2) \ 14 | - 0.25 * D * (D-1) * np.log(np.pi) \ 15 | - gammaln(0.5 * (nu - np.arange(D))).sum() 16 | 17 | def log_dirichlet_C(alpha): 18 | return gammaln(np.sum(alpha)) - gammaln(alpha).sum() 19 | 20 | class BaseGaussianMixture(): 21 | """Abstract class for mixture models.""" 22 | 23 | def __init__(self, K, init_param, seed, max_iter, display, plot_period): 24 | self.K = K 25 | self.init_param = init_param 26 | self.rd = np.random.RandomState(seed) 27 | self.max_iter = max_iter 28 | self.display = display 29 | self.plot_period = plot_period or max_iter // 10 30 | 31 | def _initialize_parameters(self, X): 32 | n_samples, D = X.shape 33 | if self.init_param == "random": 34 | resp = self.rd.rand(n_samples, self.K) 35 | resp /= resp.sum(axis=1)[:, np.newaxis] 36 | 37 | elif self.init_param == "kmeans": 38 | resp = np.zeros((n_samples, self.K)) 39 | label = KMeans(n_clusters=self.K, n_init=1).fit(X).labels_ 40 | resp[np.arange(n_samples), label] = 1 41 | 42 | elif self.init_param == "_debug": 43 | resp = np.loadtxt('data/_resp.txt') 44 | 45 | else: 46 | raise ValueError("Correct values for 'init_param' are ['random', 'kmeans']") 47 | # np.savetxt('data/_resp.txt', resp) 48 | 49 | self._initialize(X, resp) 50 | 51 | @abstractmethod 52 | def _initalize(self, X, resp): 53 | """Initialize the model parameters and hyperparameters""" 54 | pass 55 | 56 | def fit(self, X): 57 | self.fit_predict(X) 58 | return self 59 | 60 | @abstractmethod 61 | def fit_predict(self, X): 62 | pass 63 | 64 | @abstractmethod 65 | def _get_final_parameters(self): 66 | pass 67 | 68 | def display_2D(self, X, n_levels=21, show_components=True): 69 | assert X.shape[1] == 2, "Only 2D display is available" 70 | ## Grid 71 | xmin, xmax, ymin, ymax = X[:,0].min(), X[:,0].max(), X[:,1].min(), X[:,1].max() 72 | mx, my =.1 * (xmax - xmin), .1 * (ymax - ymin) # margins 73 | xmin, xmax, ymin, ymax = xmin - mx, xmax + mx, ymin - my, ymax + my 74 | # xmin, xmax = min(X[:,0].min(), X[:,1].min()), max(X[:,0].max(), X[:,1].max()) 75 | # ymin, ymax = xmin, xmax 76 | x = np.linspace(xmin, xmax, 200) 77 | y = np.linspace(ymin, ymax, 200) 78 | x, y = np.meshgrid(x, y) 79 | pos = np.empty(x.shape + (2,)) 80 | pos[:,:,0] = x; pos[:,:,1] = y 81 | rvs = [multivariate_normal(self.m[k], self.covs[k]) for k in range(self.K)] 82 | Z = sum([self.weights[k] * rvs[k].pdf(pos) for k in range(self.K)]) 83 | 84 | plt.figure(figsize=(6,6)) 85 | 86 | ## Heatmap 87 | scale = np.amax(np.abs(Z)) 88 | plt.imshow(Z, interpolation='bilinear', origin='lower', vmin=-scale,vmax=scale, 89 | cmap="RdBu", extent=(xmin, xmax, ymin, ymax)) 90 | 91 | ## Log-contours 92 | levels_exp = np.linspace(np.floor(np.log10(Z.min())-1), np.ceil(np.log10(Z.max())+1), n_levels) 93 | levels = np.power(10, levels_exp) 94 | plt.contour(x, y, Z, linewidths=1., colors="#93b8db", 95 | levels=levels, norm=LogNorm()) # A1B5C8 96 | 97 | ## Scatter plot of the dataset 98 | plt.plot(*X.T, '.', c='k', alpha=0.6) 99 | plt.xlim(xmin, xmax) 100 | plt.ylim(ymin, ymax) 101 | 102 | ## Display each component of the GM 103 | if show_components: 104 | ax = plt.gca() 105 | for k in range(self.K): 106 | if self.weights[k] >= 1e-5: 107 | plot_confidence_ellipse(self.m[k], self.covs[k], 0.9, ax=ax, 108 | ec='brown', linestyle=(0, (5, 2)), 109 | alpha=max(0.3, self.weights[k] / max(self.weights))) -------------------------------------------------------------------------------- /src/viGMM_CB.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import matplotlib.pyplot as plt 3 | from .base import BaseGaussianMixture, log_wishart_B 4 | from sklearn.cluster import KMeans 5 | from scipy.special import digamma, gammaln, logsumexp 6 | from utils import plot_confidence_ellipse 7 | from scipy.stats import multivariate_normal 8 | from matplotlib.colors import LogNorm 9 | 10 | class VariationalGaussianMixtureCB(BaseGaussianMixture): 11 | """Variarional Bayesian estimation of a Gaussian mixture 12 | 13 | References 14 | ---------- 15 | [1] Corduneanu, Adrian and Bishop, Christopher M. (2001), "Variational 16 | Bayesian Model Selection for Mixture Distributions", in Proc. AI and 17 | Statistics Conf., pp. 27-34.""" 18 | 19 | def __init__(self, K, init_param="random", seed=2208, max_iter=200, 20 | beta0=None, nu0=None, invW0=None, 21 | display=False, plot_period=None): 22 | super().__init__( 23 | K, init_param=init_param, seed=seed, max_iter=max_iter, 24 | display=display, plot_period=plot_period) 25 | 26 | self.beta0 = beta0 27 | self.nu0 = nu0 28 | self.invW0 = invW0 29 | 30 | def _initialize(self, X, resp): 31 | n_samples, D = X.shape 32 | self.nu0 = self.nu0 or D 33 | self.invW0 = self.invW0 or np.atleast_2d(np.cov(X.T)) 34 | self.beta0 = self.beta0 or 1. 35 | 36 | self.weights = np.ones(self.K) / self.K 37 | 38 | ## Init expectations 39 | expect = {"T" : np.array([self.nu0 * np.linalg.inv(self.invW0) for _ in range(self.K)]), 40 | "log_det_T" : np.zeros(self.K), 41 | "mu" : np.zeros((self.K, D)), # (23) 42 | "mu_muT" : np.zeros((self.K, D, D))} 43 | 44 | self._update_params(X, resp, expect) 45 | 46 | def fit_predict(self, X): 47 | _, D = X.shape 48 | self._initialize_parameters(X) 49 | 50 | self.elbo = np.empty(self.max_iter) 51 | for i in range(self.max_iter): 52 | expect = self._compute_expectations(D) 53 | log_resp, log_rho_tilde = self._compute_resp(X, expect) 54 | resp = np.exp(log_resp) 55 | self._update_params(X, resp, expect) 56 | self._m_step(resp) 57 | self.elbo[i] = self._compute_lower_bound(X, log_resp, resp, log_rho_tilde, expect) 58 | 59 | if self.display and D == 2 and i % self.plot_period == 0: 60 | self._get_final_parameters() 61 | self.display_2D(X) 62 | plt.title(f'iteration {i}') 63 | plt.show() 64 | 65 | self._get_final_parameters() 66 | if self.display and D == 2: 67 | self.display_2D(X) 68 | plt.title(f'iteration {i}') 69 | plt.show() 70 | 71 | def _compute_expectations(self, D): 72 | exp_T = self.nu[:, np.newaxis, np.newaxis] * np.linalg.inv(self.invW) # (25) 73 | exp_log_det_T = np.sum(digamma(0.5 * (self.nu - np.arange(D)[:,np.newaxis])), axis=0) \ 74 | + D * np.log(2) - np.log(np.linalg.det(self.invW)) # (26) 75 | 76 | expect = {"T" : exp_T, 77 | "log_det_T" : exp_log_det_T, 78 | "mu" : np.copy(self.m), # (23) 79 | "mu_muT" : np.zeros_like(self.S), 80 | "muT_mu": np.zeros(self.K)} 81 | 82 | invS = np.linalg.inv(self.S) 83 | for k in range(self.K): 84 | expect["mu_muT"][k] = invS[k] + np.outer(self.m[k], self.m[k]) # (24) 85 | expect["muT_mu"][k] = np.trace(invS[k]) + self.m[k] @ self.m[k] 86 | return expect 87 | 88 | def _compute_resp(self, X, expect): 89 | N, D = X.shape 90 | log_rho_tilde = np.zeros((N, self.K)) 91 | for n in range(N): 92 | for k in range(self.K): 93 | log_rho_tilde[n, k] = 0.5 * expect["log_det_T"][k] - 0.5 * np.trace( 94 | expect["T"][k] @ (np.outer(X[n], X[n]) - np.outer(X[n], expect["mu"][k]) \ 95 | - np.outer(expect["mu"][k], X[n]) + expect["mu_muT"][k]) 96 | ) # (17) 97 | log_rho = log_rho_tilde + np.log(self.weights + 10 * np.finfo(self.weights.dtype).eps) 98 | log_resp = log_rho - logsumexp(log_rho, axis=1)[:, np.newaxis] # (16) 99 | return log_resp, log_rho_tilde 100 | 101 | def _update_params(self, X, resp, expect): 102 | N, D = X.shape 103 | eta = resp.sum(axis=0) + 10*np.finfo(resp.dtype).eps # sum_n (z_nk) 104 | self.nu = self.nu0 + eta # (20) 105 | self.S = self.beta0 * np.eye(D) + expect["T"] * eta[:,np.newaxis,np.newaxis] # (18) 106 | invS = np.linalg.inv(self.S) 107 | 108 | self.invW = np.zeros((self.K, D, D)) # init (21) 109 | self.m = np.zeros((self.K, D)) # init (19) 110 | 111 | for k in range(self.K): 112 | self.m[k] = invS[k] @ expect["T"][k] @ (resp[:, k] @ X) # (19) 113 | 114 | s = np.zeros((D, D)) 115 | for n in range(N): 116 | s += resp[n, k] * (np.outer(X[n], X[n]) - np.outer(X[n], expect["mu"][k]) \ 117 | - np.outer(expect["mu"][k], X[n]) + expect["mu_muT"][k]) # (21 --) 118 | self.invW[k] = self.invW0 + s # (-- 21) 119 | 120 | def _m_step(self, resp): 121 | self.weights = resp.sum(axis=0) / resp.sum() 122 | 123 | def _compute_lower_bound(self, X, log_resp, resp, log_rho_tilde, expect): 124 | N, D = X.shape 125 | 126 | ln_p_x = np.sum(resp * (log_rho_tilde)) # (28) 127 | ln_p_z = np.sum(resp * np.log(self.weights)) # (29) 128 | ln_p_mu = self.K * D * np.log(0.5 * self.beta0 / np.pi) \ 129 | - 0.5 * self.beta0 * np.sum(expect["muT_mu"]) # (30) 130 | ln_p_T = self.K * log_wishart_B(self.invW0, self.nu0) \ 131 | + 0.5 * (self.nu0 - D - 1) * expect["log_det_T"].sum() \ 132 | - 0.5 * np.trace(self.invW0 * expect["T"].sum()) # (31) 133 | 134 | ln_q_z = np.sum(resp * np.log(resp)) # (32) 135 | ln_q_mu = - 0.5 * self.K * D * (1 + np.log(2 * np.pi)) \ 136 | + 0.5 * np.sum(np.log(np.linalg.det(self.S))) # (33) 137 | ln_q_T = np.sum([log_wishart_B(self.invW[k], self.nu[k]) for k in range(self.K)]) \ 138 | + np.sum(0.5 * (self.nu - D - 1) * expect['log_det_T']) \ 139 | - np.sum(0.5 * np.trace(self.invW @ expect['T'], axis1=1, axis2=2)) # (34) 140 | return ln_p_x + ln_p_z + ln_p_mu + ln_p_T + ln_q_z + ln_q_mu + ln_q_T 141 | 142 | def _get_final_parameters(self): 143 | self.covs = self.invW / self.nu[:, np.newaxis, np.newaxis] -------------------------------------------------------------------------------- /src/viGMM_full.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import matplotlib.pyplot as plt 3 | from .base import BaseGaussianMixture, log_wishart_B, log_dirichlet_C 4 | from scipy.special import digamma, gammaln, logsumexp 5 | from utils import plot_confidence_ellipse 6 | 7 | 8 | class VariationalGaussianMixture(BaseGaussianMixture): 9 | """Variational Bayesian estimation of a Gaussian mixture 10 | 11 | References 12 | ---------- 13 | [2] Bishop, Christopher M. (2006). "Pattern recognition and machine 14 | learning". Vol. 4 No. 4. New York: Springer.""" 15 | 16 | def __init__(self, K, init_param="random", seed=2208, max_iter=200, 17 | alpha0=None, m0=None, beta0=None, invW0=None, nu0=None, 18 | display=False, plot_period=None): 19 | super().__init__( 20 | K, init_param=init_param, seed=seed, max_iter=max_iter, 21 | display=display, plot_period=plot_period) 22 | self.alpha0 = alpha0 23 | self.beta0 = beta0 24 | self.m0 = m0 25 | self.nu0 = nu0 26 | self.invW0 = invW0 27 | 28 | def _initialize(self, X, resp): 29 | n_samples, D = X.shape 30 | self.alpha0 = self.alpha0 or (1. / self.K) 31 | self.beta0 = self.beta0 or 1. 32 | self.m0 = self.m0 or X.sum(axis=0) 33 | self.nu0 = self.nu0 or D 34 | self.invW0 = self.invW0 or np.atleast_2d(np.cov(X.T)) 35 | 36 | self._m_step(X, np.log(resp)) 37 | 38 | def fit_predict(self, X): 39 | """Estimate model parameters""" 40 | _, D = X.shape 41 | self._initialize_parameters(X) 42 | 43 | self.elbo = np.empty(self.max_iter) 44 | for i in range(self.max_iter): 45 | ln_resp, ln_lambda_tilde, ln_pi_tilde = self._e_step(X) 46 | self._m_step(X, ln_resp) 47 | self.elbo[i] = self._compute_lower_bound(X, ln_resp, ln_lambda_tilde, ln_pi_tilde) 48 | 49 | if self.display and D == 2 and i % self.plot_period == 0: 50 | self._get_final_parameters() 51 | self.display_2D(X) 52 | plt.title(f'iteration {i}') 53 | plt.show() 54 | 55 | self._get_final_parameters() 56 | if self.display and D == 2: 57 | self.display_2D(X) 58 | plt.title(f'iteration {i}') 59 | plt.show() 60 | # Final e-step to guarantee that the labels are consistent 61 | ln_resp, *_ = self._e_step(X) 62 | return ln_resp.argmax(axis=1) 63 | 64 | def _get_final_parameters(self): 65 | self.weights = self.alpha / np.sum(self.alpha) 66 | self.covs = self.invW / self.nu[:, np.newaxis, np.newaxis] 67 | 68 | def _compute_statististics(self, X, resp): 69 | _, D = X.shape 70 | N = resp.sum(axis=0) + 10*np.finfo(resp.dtype).eps # (10.51) 71 | x_bar = (resp.T @ X) / N[:, np.newaxis] # (10.52) 72 | S = np.zeros((self.K, D, D)) 73 | for k in range(self.K): 74 | Xc = X - x_bar[k] 75 | S[k] = ((resp[:,k] * Xc.T) @ Xc) / N[k] # (10.53) 76 | S[k].flat[::D+1] += 1e-6 # regularization added to the diag. Assure that the covariance matrices are all positive 77 | return N, x_bar, S 78 | 79 | 80 | def _e_step(self, X): 81 | n_samples, D = X.shape 82 | W = np.linalg.inv(self.invW) 83 | 84 | E = np.zeros((n_samples, self.K)) 85 | for k in range(self.K): 86 | Xc = X - self.m[k] 87 | E[:,k] = D / self.beta[k] + self.nu[k] * np.sum(Xc @ W[k] * Xc, axis=1) # (10.64) 88 | ln_lambda_tilde = np.sum(digamma(0.5 * (self.nu - np.arange(0, D)[:,np.newaxis])), axis=0) \ 89 | + D * np.log(2) + np.log(np.linalg.det(W)) # (10.65) 90 | ln_pi_tilde = digamma(self.alpha) - digamma(np.sum(self.alpha)) # (10.66) 91 | 92 | ln_rho = ln_pi_tilde + 0.5*ln_lambda_tilde - 0.5 * (E + D * np.log(2 * np.pi)) # (10.46) 93 | ln_resp = ln_rho - np.c_[logsumexp(ln_rho, axis=1)] # (10.49) 94 | return ln_resp, ln_lambda_tilde, ln_pi_tilde 95 | 96 | def _m_step(self, X, ln_resp): 97 | _, D = X.shape 98 | N, x_bar, S = self._compute_statististics(X, np.exp(ln_resp)) 99 | 100 | self.alpha = self.alpha0 + N # (10.58), weight concentration 101 | self.beta = self.beta0 + N # (10.60), mean precision 102 | self.nu = self.nu0 + N # (10.63), degrees of freedom 103 | self.m = (self.beta0 * self.m0 + x_bar * N[:,np.newaxis]) / self.beta[:,np.newaxis] # (10.61), means 104 | 105 | self.invW = np.zeros((self.K, D, D)) # covariances 106 | for k in range(self.K): 107 | xc = x_bar[k] - self.m0 108 | self.invW[k] = self.invW0 + N[k] * S[k] + (self.beta0 * N[k]) * ( 109 | np.outer(xc, xc) / self.beta[k]) # (10.62) 110 | 111 | def _compute_lower_bound(self, X, ln_resp, ln_lambda_tilde, ln_pi_tilde): 112 | _, D = X.shape 113 | resp = np.exp(ln_resp) 114 | N, x_bar, S = self._compute_statististics(X, resp) 115 | W = np.linalg.inv(self.invW) 116 | 117 | ln_p_x = 0.5 * np.sum(N * ln_lambda_tilde) \ 118 | - 0.5 * D * np.sum(N / self.beta) \ 119 | - 0.5 * np.sum(N * self.nu * np.trace(S @ W, axis1=1, axis2=2)) \ 120 | - 0.5 * np.sum([N[k] * self.nu[k] * (x_bar[k] - self.m[k]) @ W[k] @ (x_bar[k] - self.m[k]) for k in range(self.K)]) \ 121 | - 0.5 * N.sum() * D * np.log(2 * np.pi)# (10.71) 122 | ln_p_z = np.sum(resp * ln_pi_tilde) # (10.72) 123 | ln_p_pi = (self.alpha0 - 1) * ln_pi_tilde.sum() + log_dirichlet_C([self.alpha0] * self.K) # (10.73) 124 | ln_p_mu_lambda = 0.5 * np.sum(ln_lambda_tilde) \ 125 | + 0.5 * self.K * D * np.log(0.5 * self.beta0 / np.pi) \ 126 | - 0.5 * D * (self.beta0 / self.beta).sum() \ 127 | - 0.5 * self.beta0 * np.sum([self.nu[k] * (self.m[k] - self.m0) @ W[k] @ (self.m[k] - self.m0) for k in range(self.K)]) \ 128 | + self.K * log_wishart_B(self.invW0, self.nu0) \ 129 | + 0.5 * (self.nu0 - D - 1) * ln_lambda_tilde.sum() \ 130 | - 0.5 * np.sum(self.nu * np.trace(self.invW0 @ W, axis1=1, axis2=2)) # (10.74) 131 | 132 | ln_q_z = np.sum(resp * ln_resp) # (10.75) 133 | ln_q_pi = np.sum((self.alpha - 1) * ln_pi_tilde) + log_dirichlet_C(self.alpha) # (10.76) 134 | ln_q_mu_lambda = 0.5 * np.sum(ln_lambda_tilde) - 0.5 * self.K * D \ 135 | + np.sum(0.5 * D * np.log(0.5 * self.beta / np.pi)) \ 136 | + np.sum([log_wishart_B(self.invW[k], self.nu[k]) for k in range(self.K)]) \ 137 | + np.sum(0.5 * (self.nu - D - 1) * ln_lambda_tilde) \ 138 | - np.sum(0.5 * self.nu * D) # (10.77) 139 | 140 | return ln_p_x + ln_p_z + ln_p_pi + ln_p_mu_lambda - ln_q_z - ln_q_pi - ln_q_mu_lambda 141 | 142 | 143 | # def _display_2D(self, X, i=None, **kwargs): 144 | # assert X.shape[1] == 2, "Only 2D display is available" 145 | # plt.figure() 146 | # plt.plot(*X.T, 'o', c='dimgrey', alpha=0.5) 147 | # ax = plt.gca() 148 | # for k in range(self.K): 149 | # if not(np.allclose(self.m[k], [0, 0], atol=1e-3) and self.weights[k] < 1e-3): 150 | # plot_confidence_ellipse(self.m[k], self.covs[k], 0.9, ax=ax, ec='teal') 151 | # if i: plt.title(f'iteration {i}') 152 | # plt.show() 153 | -------------------------------------------------------------------------------- /utils.py: -------------------------------------------------------------------------------- 1 | 2 | #!/usr/bin/env python3 3 | # Julien Nonin 4 | 5 | #%% 6 | import numpy as np 7 | from numpy import linalg as la 8 | import matplotlib.pyplot as plt 9 | from matplotlib.patches import Ellipse 10 | 11 | def plot_confidence_ellipse(mu, cov, alph, ax, clabel=None, label_bg='white', **kwargs): 12 | """Display a confidence ellipse of a bivariate normal distribution 13 | 14 | Arguments: 15 | mu {array-like of shape (2,)} -- mean of the distribution 16 | cov {array-like of shape(2,2)} -- covariance matrix 17 | alph {float btw 0 and 1} -- level of confidence 18 | ax {plt.Axes} -- axes on which to display the ellipse 19 | clabel {str} -- label to add to ellipse (default: {None}) 20 | label_bg {str} -- background of clabel's textbox 21 | kwargs -- other arguments given to class Ellipse 22 | """ 23 | c = -2 * np.log(1 - alph) # quantile at alpha of the chi_squarred distr. with df = 2 24 | Lambda, Q = la.eig(cov) # eigenvalues and eigenvectors (col. by col.) 25 | 26 | ## Compute the attributes of the ellipse 27 | width, heigth = 2 * np.sqrt(c * Lambda) 28 | # compute the value of the angle theta (in degree) 29 | theta = 180 * np.arctan(Q[1,0] / Q[0,0]) / np.pi if cov[1,0] else 0 30 | 31 | ## Create the ellipse 32 | if 'fc' not in kwargs.keys(): 33 | kwargs['fc'] = 'None' 34 | level_line = Ellipse(mu, width, heigth, angle=theta, **kwargs) 35 | 36 | ## Display a label 'clabel' on the ellipse 37 | if clabel: 38 | col = kwargs['ec'] if 'ec' in kwargs.keys() and kwargs['ec'] != 'None' else 'black' # color of the text 39 | pos = Q[:,1] * np.sqrt(c * Lambda[1]) + mu # position along the heigth 40 | 41 | ax.text(*pos, clabel, color=col, 42 | rotation=theta, ha='center', va='center', rotation_mode='anchor', # rotation 43 | bbox=dict(boxstyle='round',ec='None',fc=label_bg, alpha=1)) # white box 44 | 45 | return ax.add_patch(level_line) 46 | 47 | if __name__ == '__main__': 48 | N = 100 49 | mu = np.array([0, 1]) 50 | cov = np.array([[0.30, 0.03], [0.03, 0.15]]) 51 | np.random.seed(0) 52 | x = np.random.multivariate_normal(mu, cov, N) 53 | 54 | plt.figure() 55 | ax = plt.gca() 56 | plot_confidence_ellipse(mu, cov, 0.99, ax, ec='crimson', clabel='99 %') 57 | plot_confidence_ellipse(mu, cov, 0.95, ax, ec='r') 58 | plot_confidence_ellipse(mu, cov, 0.90, ax, ec='orange', clabel='90 %') 59 | # plot_confidence_ellipse(mu, cov, 1-np.exp(-3**2/2), plt.gca(), fc='lightblue', ec='darkblue', alpha=0.3, label=r'3$\sigma$') 60 | plt.plot(*x.T, 'o', c='dimgrey') 61 | ax.plot(*mu, 'kx') 62 | plt.axis('equal') 63 | plt.show() 64 | -------------------------------------------------------------------------------- /viGMM_CB.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import numpy as np\n", 10 | "import matplotlib.pyplot as plt\n", 11 | "from scipy.special import digamma, logsumexp\n", 12 | "from utils import plot_confidence_ellipse" 13 | ] 14 | }, 15 | { 16 | "cell_type": "code", 17 | "execution_count": null, 18 | "metadata": {}, 19 | "outputs": [], 20 | "source": [ 21 | "X = np.loadtxt('data/gaussian.txt')\n", 22 | "X = (X - X.mean(axis=0)) / X.std(axis=0)\n", 23 | "N, D = X.shape # n_samples, n_features" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": null, 29 | "metadata": {}, 30 | "outputs": [], 31 | "source": [ 32 | "# Create a synthetic data set made up with 5 Gaussian components\n", 33 | "m_true = np.array([[0, 0], [3, -3], [3, 3], [-3, 3], [-3, -3]])\n", 34 | "covs_true = np.array([[[1, 0], [0, 1]], [[1, 0.5], [0.5, 1]], [[1, -0.5], [-0.5, 1]], [[1, 0.5], [0.5, 1]], [[1, -0.5], [-0.5, 1]]])\n", 35 | "X = np.concatenate([np.random.multivariate_normal(m_true[k], covs_true[k], 100) for k in range(len(m_true))])\n", 36 | "N, D = X.shape\n", 37 | "\n", 38 | "plt.figure(figsize=(5,5))\n", 39 | "ax = plt.gca()\n", 40 | "for k in range(len(m_true)):\n", 41 | " plot_confidence_ellipse(m_true[k], covs_true[k], 0.9, ax=ax, ec='teal')\n", 42 | "plt.plot(*X.T, '.')\n", 43 | "ax.set_aspect('equal')" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "execution_count": null, 49 | "metadata": {}, 50 | "outputs": [], 51 | "source": [ 52 | "K = 5\n", 53 | "\n", 54 | "m0 = np.mean(X, axis=0)\n", 55 | "# m = np.array([[-0.49813282, -0.78529033],\n", 56 | "# [0.3 , 0.3],\n", 57 | "# [ 0.16320506, -0.01819425]])\n", 58 | "# [-0.47266126, -0.68203894],\n", 59 | "# [ 0.38457508, 0.32323383],\n", 60 | "# [ 0.22959118, 0.10605733],\n", 61 | "# [ 0.06409906, 0.27213673],\n", 62 | "# [ 0.28301891, 0.29796192],\n", 63 | "# [ 0.13476758, -0.25260453],\n", 64 | "# [ 0.28530035, -0.00930441]])\n", 65 | "m = np.zeros((K, D))\n", 66 | "\n", 67 | "# beta0 = 1\n", 68 | "# beta = beta0 * np.ones(K)\n", 69 | "\n", 70 | "invW0 = np.cov(X.T)\n", 71 | "W0 = np.linalg.inv(invW0)\n", 72 | "W = np.array([np.cov(X[np.random.choice(N, 10)].T) for _ in range(K)])\n", 73 | "# invW = np.array([invW0 for _ in range(K)])\n", 74 | "# W = np.linalg.inv(invW)\n", 75 | "# W0 = np.linalg.inv(invW0)\n", 76 | "invW = np.linalg.inv(W)\n", 77 | "\n", 78 | "beta0 = 1\n", 79 | "invS0 = np.linalg.inv(beta0 * np.eye(D))\n", 80 | "invS = np.array([invS0 for _ in range(K)])\n", 81 | "# S = np.linalg.inv(invS)\n", 82 | "# S0 = np.linalg.inv(invS0)\n", 83 | "\n", 84 | "v0 = D\n", 85 | "v = v0 * np.ones(K)\n", 86 | "\n", 87 | "pi = np.ones(K) / K\n", 88 | "\n", 89 | "resp = np.random.rand(N, K)\n", 90 | "resp /= resp.sum(axis=1)[:, np.newaxis]" 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": null, 96 | "metadata": {}, 97 | "outputs": [], 98 | "source": [ 99 | "def init():\n", 100 | " global invW0, m0, beta0, S0, v0, pi, resp\n", 101 | " m0 = np.mean(X, axis=0)\n", 102 | "# m = X[np.random.choice(N, K)]\n", 103 | " invW0 = np.cov(X.T)\n", 104 | "# W = np.array([np.cov(X[np.random.choice(N, 10)].T) for _ in range(K)])\n", 105 | "# invW = np.linalg.inv(W)\n", 106 | " beta0 = 1\n", 107 | " S0 = np.linalg.inv(beta0 * np.eye(D))\n", 108 | "# invS = np.array([invS0 for _ in range(K)])\n", 109 | " v0 = D\n", 110 | "# v = v0 * np.ones(K)\n", 111 | " pi = np.ones(K) / K\n", 112 | " resp = np.random.rand(N, K)\n", 113 | " resp /= resp.sum(axis=1)[:, np.newaxis]\n", 114 | " resp = np.loadtxt('data/resp.txt')" 115 | ] 116 | }, 117 | { 118 | "cell_type": "code", 119 | "execution_count": null, 120 | "metadata": {}, 121 | "outputs": [], 122 | "source": [ 123 | "def m_step():\n", 124 | " global pi\n", 125 | " pi = resp.sum(axis=0) / resp.sum()" 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "execution_count": null, 131 | "metadata": {}, 132 | "outputs": [], 133 | "source": [ 134 | "def compute_esp():\n", 135 | " global esp_T, esp_log_det_T, esp_mu, esp_mu_muT\n", 136 | " W = np.linalg.inv(invW)\n", 137 | " invS = np.linalg.inv(S)\n", 138 | " esp_T = np.zeros_like(W)\n", 139 | " esp_log_det_T = np.zeros(K)\n", 140 | " esp_mu = np.copy(m)\n", 141 | " esp_mu_muT = np.zeros_like(S)\n", 142 | " for k in range(K):\n", 143 | " esp_T[k] = v[k] * W[k]\n", 144 | " esp_log_det_T[k] = digamma(0.5*(v[k] - np.arange(D))).sum() + D * np.log(2) - np.log(np.linalg.det(invW[k]))\n", 145 | " esp_mu_muT[k] = invS[k] + np.outer(m[k], m[k])" 146 | ] 147 | }, 148 | { 149 | "cell_type": "code", 150 | "execution_count": null, 151 | "metadata": {}, 152 | "outputs": [], 153 | "source": [ 154 | "def compute_resp():\n", 155 | " global resp\n", 156 | " log_rho = np.zeros((N, K))\n", 157 | " for n in range(N):\n", 158 | " for k in range(K):\n", 159 | " log_rho[n, k] = np.log(pi[k]+1e-15) + 0.5 * esp_log_det_T[k] - 0.5 * np.trace(\n", 160 | " esp_T[k] @ (np.outer(X[n], X[n]) - np.outer(X[n], esp_mu[k]) - np.outer(esp_mu[k], X[n]) + esp_mu_muT[k])\n", 161 | " )\n", 162 | " log_resp = log_rho - logsumexp(log_rho, axis=1)[:, np.newaxis]\n", 163 | " resp = np.exp(log_resp)" 164 | ] 165 | }, 166 | { 167 | "cell_type": "code", 168 | "execution_count": null, 169 | "metadata": {}, 170 | "outputs": [], 171 | "source": [ 172 | "def update_params():\n", 173 | " global m, S, v, invW\n", 174 | "# m = np.zeros_like(m)\n", 175 | " S = np.zeros((K, D, D))\n", 176 | "# S_old = np.copy(S)\n", 177 | "# invS = np.zeros_like(invS)\n", 178 | "# v = np.zeros_like(v)\n", 179 | " invW = np.zeros((K, D, D))\n", 180 | " m = np.zeros((K, D))\n", 181 | " \n", 182 | " eta = resp.sum(axis=0) + 10*np.finfo(resp.dtype).eps\n", 183 | " v = v0 + eta\n", 184 | " S = esp_T * eta[:,np.newaxis,np.newaxis] + beta0 * np.eye(D)\n", 185 | " invS = np.linalg.inv(S)\n", 186 | " \n", 187 | "\n", 188 | " for k in range(K):\n", 189 | "# S[k] = beta0 * np.eye(D) + esp_T[k] * eta[k]\n", 190 | "\n", 191 | " m[k] = invS[k] @ esp_T[k] @ (resp[:, k] @ X)\n", 192 | "\n", 193 | " s = np.zeros((D, D))\n", 194 | " for n in range(N):\n", 195 | " s += resp[n, k] * (np.outer(X[n], X[n]) - np.outer(X[n], esp_mu[k]) - np.outer(esp_mu[k], X[n]) + esp_mu_muT[k])\n", 196 | " invW[k] = invW0 + s" 197 | ] 198 | }, 199 | { 200 | "cell_type": "code", 201 | "execution_count": null, 202 | "metadata": {}, 203 | "outputs": [], 204 | "source": [ 205 | "def display():\n", 206 | " plt.figure(figsize=(6,6))\n", 207 | " plt.plot(*X.T, 'o', c='dimgrey', alpha = 0.5)\n", 208 | " ax = plt.gca()\n", 209 | " for k in range(K):\n", 210 | " if pi[k] >= 1/(2*K):\n", 211 | " plot_confidence_ellipse(m[k], invW[k]/v[k], 0.9, ax=ax, ec='teal')\n", 212 | " ax.set_aspect('equal')\n", 213 | " plt.show()" 214 | ] 215 | }, 216 | { 217 | "cell_type": "code", 218 | "execution_count": null, 219 | "metadata": {}, 220 | "outputs": [], 221 | "source": [ 222 | "K = 10\n", 223 | "init()\n", 224 | "# compute_esp()\n", 225 | "esp_T = np.array([v0 * np.linalg.inv(invW0) for _ in range(K)])\n", 226 | "esp_mu = np.zeros((K, D))\n", 227 | "esp_mu_muT = np.zeros((K, D, D))\n", 228 | "update_params()\n", 229 | "display()\n", 230 | "# compute_exp()\n", 231 | "# update_params()\n", 232 | "# update_params()\n", 233 | "# display()\n", 234 | "\n", 235 | "for i in range(300):\n", 236 | " compute_esp()\n", 237 | " compute_resp()\n", 238 | " update_params()\n", 239 | " m_step()\n", 240 | " if i%10 == 0:\n", 241 | " print(i)\n", 242 | " display()" 243 | ] 244 | } 245 | ], 246 | "metadata": { 247 | "kernelspec": { 248 | "display_name": "Python 3", 249 | "language": "python", 250 | "name": "python3" 251 | }, 252 | "language_info": { 253 | "codemirror_mode": { 254 | "name": "ipython", 255 | "version": 3 256 | }, 257 | "file_extension": ".py", 258 | "mimetype": "text/x-python", 259 | "name": "python", 260 | "nbconvert_exporter": "python", 261 | "pygments_lexer": "ipython3", 262 | "version": "3.7.6" 263 | } 264 | }, 265 | "nbformat": 4, 266 | "nbformat_minor": 4 267 | } 268 | -------------------------------------------------------------------------------- /viGMM_McGrory.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": null, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import numpy as np\n", 10 | "import matplotlib.pyplot as plt\n", 11 | "from sklearn.mixture import BayesianGaussianMixture\n", 12 | "from sklearn.cluster import KMeans\n", 13 | "from scipy.special import digamma, logsumexp\n", 14 | "from utils import plot_confidence_ellipse\n" 15 | ] 16 | }, 17 | { 18 | "cell_type": "code", 19 | "execution_count": null, 20 | "metadata": {}, 21 | "outputs": [], 22 | "source": [ 23 | "X = np.loadtxt('data/gaussian.txt')\n", 24 | "X = (X - X.mean(axis=0)) / X.std(axis=0)\n", 25 | "N, D = X.shape # n_samples, n_features" 26 | ] 27 | }, 28 | { 29 | "cell_type": "code", 30 | "execution_count": null, 31 | "metadata": {}, 32 | "outputs": [], 33 | "source": [ 34 | "m_true = np.array([[0, 0], [3, -3], [3, 3], [-3, 3], [-3, -3]])\n", 35 | "covs_true = np.array([[[1, 0], [0, 1]], [[1, 0.5], [0.5, 1]], [[1, -0.5], [-0.5, 1]], [[1, 0.5], [0.5, 1]], [[1, -0.5], [-0.5, 1]]])\n", 36 | "X = np.concatenate([np.random.multivariate_normal(m_true[k], covs_true[k], 100) for k in range(len(m_true))])\n", 37 | "N, D = X.shape\n", 38 | "\n", 39 | "plt.figure(figsize=(5,5))\n", 40 | "ax = plt.gca()\n", 41 | "for k in range(len(m_true)):\n", 42 | " plot_confidence_ellipse(m_true[k], covs_true[k], 0.9, ax=ax, ec='teal')\n", 43 | "plt.plot(*X.T, '.')\n", 44 | "ax.set_aspect('equal')" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "def init():\n", 54 | "# global invW0, invW, resp, m, m0, beta0, invS0, invS, v0, v, pi\n", 55 | " global m0, invW0, beta0, v0, alpha0, resp\n", 56 | " m0 = X.sum(axis=0)\n", 57 | "# m = X[np.random.choice(N, K)]\n", 58 | " invW0 = np.cov(X.T)\n", 59 | "# W = np.array([np.cov(X[np.random.choice(N, 10)].T) for _ in range(K)])\n", 60 | "# invW = np.linalg.inv(W)\n", 61 | " beta0 = 1.\n", 62 | "# invS0 = np.linalg.inv(beta0 * np.eye(D))\n", 63 | "# invS = np.array([invS0 for _ in range(K)])\n", 64 | " v0 = D\n", 65 | "# v = v0 * np.ones(K)\n", 66 | "# pi = np.ones(K) / K\n", 67 | " alpha0 = 1 / K\n", 68 | " \n", 69 | " resp = np.random.rand(N, K)\n", 70 | " resp /= resp.sum(axis=1)[:, np.newaxis]\n", 71 | " \n", 72 | " resp = np.zeros((N, K))\n", 73 | " label = KMeans(n_clusters= K, n_init=1).fit(X).labels_\n", 74 | " resp[np.arange(N), label] = 1\n", 75 | " resp /= resp.sum(axis=1)[:, np.newaxis]\n", 76 | " resp = np.loadtxt('data/resp.txt')\n", 77 | " " 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": null, 83 | "metadata": {}, 84 | "outputs": [], 85 | "source": [ 86 | "def m_step():\n", 87 | " global pi\n", 88 | " pi = resp.sum(axis=0) / resp.sum()" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": null, 94 | "metadata": {}, 95 | "outputs": [], 96 | "source": [ 97 | "def compute_esp():\n", 98 | " global esp_T, esp_log_det_T, esp_mu#, esp_mu_muT\n", 99 | " global esp_log_pi\n", 100 | " W = np.linalg.inv(invW)\n", 101 | " esp_T = np.zeros_like(W)\n", 102 | " esp_log_det_T = np.zeros(K)\n", 103 | " esp_mu = np.copy(m)\n", 104 | " esp_log_pi = digamma(alpha) - digamma(alpha.sum())\n", 105 | "# esp_mu_muT = np.zeros_like(invS)\n", 106 | " for k in range(K):\n", 107 | " esp_T[k] = v[k] * W[k]\n", 108 | " esp_log_det_T[k] = digamma(0.5*(v[k] - np.arange(D))).sum() + D * np.log(2) - np.log(np.linalg.det(invW[k]))\n", 109 | "# esp_mu_muT[k] = invS[k] + np.outer(m[k], m[k])" 110 | ] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "execution_count": null, 115 | "metadata": {}, 116 | "outputs": [], 117 | "source": [ 118 | "def compute_resp():\n", 119 | " global resp\n", 120 | " log_rho = np.zeros((N, K))\n", 121 | " for n in range(N):\n", 122 | " for k in range(K):\n", 123 | " log_rho[n, k] = esp_log_pi[k] + 0.5 * esp_log_det_T[k] - 0.5 * np.trace(\n", 124 | " esp_T[k] @ np.outer(X[n] - esp_mu[k], X[n] - esp_mu[k])) - 0.5 * D / beta[k] \n", 125 | " log_resp = log_rho - logsumexp(log_rho, axis=1)[:, np.newaxis]\n", 126 | " resp = np.exp(log_resp)" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": null, 132 | "metadata": {}, 133 | "outputs": [], 134 | "source": [ 135 | "def update_params():\n", 136 | " global m, v, invW, alpha, beta\n", 137 | " invW = np.zeros((K, D, D))\n", 138 | " \n", 139 | " eta = resp.sum(axis=0) + 10*np.finfo(resp.dtype).eps\n", 140 | " v = v0 + eta\n", 141 | " alpha = alpha0 + eta\n", 142 | " beta = beta0 + eta\n", 143 | " m = (beta0 * m0 + resp.T @ X) / beta[:,np.newaxis]\n", 144 | "\n", 145 | " for k in range(K):\n", 146 | " s = np.zeros((D, D))\n", 147 | " for n in range(N):\n", 148 | " s += resp[n, k] * (np.outer(X[n], X[n]))\n", 149 | " invW[k] = invW0 + s + beta0 * np.outer(m0, m0) - beta[k] * np.outer(m[k], m[k])" 150 | ] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": null, 155 | "metadata": {}, 156 | "outputs": [], 157 | "source": [ 158 | "def display():\n", 159 | " plt.figure(figsize=(6,6))\n", 160 | " plt.plot(*X.T, 'o', c='dimgrey', alpha = 0.5)\n", 161 | " ax = plt.gca()\n", 162 | " \n", 163 | " weights = alpha / alpha.sum()\n", 164 | " for k in range(K):\n", 165 | " if weights[k] >= 1/(2*K):\n", 166 | " plot_confidence_ellipse(m[k], invW[k]/v[k], 0.9, ax=ax, ec='teal')\n", 167 | "# ax.set_aspect('equal')\n", 168 | " plt.show()" 169 | ] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "execution_count": null, 174 | "metadata": {}, 175 | "outputs": [], 176 | "source": [ 177 | "K = 10\n", 178 | "init()\n", 179 | "update_params()\n", 180 | "display()\n", 181 | "\n", 182 | "for i in range(200):\n", 183 | " compute_esp()\n", 184 | " compute_resp()\n", 185 | " update_params()\n", 186 | "# m_step()\n", 187 | " if i % 20 == 0:\n", 188 | " print(i)\n", 189 | " display()" 190 | ] 191 | }, 192 | { 193 | "cell_type": "code", 194 | "execution_count": null, 195 | "metadata": {}, 196 | "outputs": [], 197 | "source": [] 198 | } 199 | ], 200 | "metadata": { 201 | "kernelspec": { 202 | "display_name": "Python 3", 203 | "language": "python", 204 | "name": "python3" 205 | }, 206 | "language_info": { 207 | "codemirror_mode": { 208 | "name": "ipython", 209 | "version": 3 210 | }, 211 | "file_extension": ".py", 212 | "mimetype": "text/x-python", 213 | "name": "python", 214 | "nbconvert_exporter": "python", 215 | "pygments_lexer": "ipython3", 216 | "version": "3.7.6" 217 | } 218 | }, 219 | "nbformat": 4, 220 | "nbformat_minor": 4 221 | } 222 | --------------------------------------------------------------------------------