├── .gitignore ├── .travis.yml ├── ChangeLog.rst ├── HACKING.rst ├── LICENSE ├── README.rst ├── doc ├── ChangeLog.rst ├── HACKING.rst ├── Makefile ├── README.rst ├── conf.py ├── filters.rst ├── index.rst └── pdfs.rst ├── pybayes ├── .gitignore ├── __init__.pxd ├── __init__.py ├── filters.pxd ├── filters.py ├── pdfs.pxd ├── pdfs.py ├── stresses │ ├── __init__.py │ ├── __main__.py │ ├── data │ │ └── stress_kalman_data.mat │ ├── stress_filters.pxd │ ├── stress_filters.py │ └── support.py ├── tests │ ├── __init__.py │ ├── __main__.py │ ├── support.py │ ├── test_filters.py │ ├── test_pdfs.py │ ├── test_wrappers_linalg.pxd │ ├── test_wrappers_linalg.py │ ├── test_wrappers_numpy.pxd │ └── test_wrappers_numpy.py └── wrappers │ ├── __init__.py │ ├── _linalg.pxd │ ├── _linalg.py │ ├── _linalg.pyx │ ├── _numpy.pxd │ ├── _numpy.py │ └── _numpy.pyx ├── setup.py ├── support ├── __init__.py ├── debian │ ├── debian.changelog │ ├── debian.control │ ├── debian.rules │ └── python-pybayes.dsc ├── dist.py ├── dist_cmd_build.py ├── dist_cmd_build_prepare.py ├── dist_cmd_stress.py ├── dist_cmd_test.py └── rpm │ └── python-pybayes.spec └── thesis ├── .gitignore ├── KF.fig ├── KF.pdf ├── KFPerf.pdf ├── benchmark_c_cy_py ├── .gitignore ├── README ├── edgy-results.ods ├── integrate_c.c ├── integrate_c.h ├── integrate_cython.c ├── integrate_cython.pyx ├── integrate_python.py ├── results-esprimo-optimised.txt ├── results-esprimo-unoptimised.txt ├── results-kmlinux-optimised.txt ├── results-kmlinux-unoptimised.txt ├── run.py ├── setup.py └── zavislost_na_gcc_prepinacich.txt ├── benchmark_c_java ├── .gitignore ├── README ├── test.c └── test.java ├── benchmark_kalman ├── big_cython.txt ├── big_matlab.txt ├── big_python.txt ├── edgy-results.ods ├── huge_cython.txt ├── huge_matlab.txt ├── huge_python.txt ├── plot.py ├── small_cython.txt ├── small_matlab.txt └── small_python.txt ├── benchmark_py_numpy ├── spectralnorm_numpy.py ├── spectralnorm_plain_1thread.py └── spectralnorm_plain_4threads.py ├── bibliography.bib ├── chap0-introduction.tex ├── chap0-notation.tex ├── chap1-theory.tex ├── chap2-software.tex ├── chap3-pybayes.tex ├── chap4-conclusion.tex ├── coverage.pdf ├── cvut-logo-bw-600.pdf ├── diagrams ├── PyBayes.dia ├── PyBayes.pdf ├── emp_pdfs.dia ├── emp_pdfs.pdf ├── filters.dia ├── filters.pdf ├── gaussian_pdfs.dia ├── gaussian_pdfs.pdf ├── page_setup.dia ├── particile_filters.dia ├── pdfs.dia ├── pdfs.pdf ├── pdfs_model_outdated.dia ├── rvs.dia └── rvs.pdf ├── prezentace ├── .gitignore ├── air-header.png ├── beamerthemeAir.sty ├── edgy-results.pdf └── prezentace.tex ├── startpages.tex ├── thesis.tex └── zadani1.lyx /.gitignore: -------------------------------------------------------------------------------- 1 | # these files are autogenerated by distutils: 2 | /MANIFEST 3 | /build 4 | /dist 5 | /doc/_build 6 | *~ 7 | *.pyc 8 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | 3 | python: 4 | - 2.7 5 | - 3.2 6 | - 3.3 7 | 8 | env: 9 | - USE_CYTHON=yes 10 | - USE_CYTHON=no 11 | 12 | install: 13 | # Ceygen needs system Eigen installed 14 | - if test ${USE_CYTHON} = yes; then pip install cython --use-mirrors; sudo apt-get install -qq libeigen3-dev; pip install ceygen --use-mirrors; fi 15 | # Python 3 environment on travis doesn't see python3-numpy package, install manually 16 | - if test ${TRAVIS_PYTHON_VERSION%%.*} = 3; then pip install numpy --use-mirrors; fi 17 | 18 | script: 19 | - python setup.py -v --use-cython=${USE_CYTHON} test stress 20 | -------------------------------------------------------------------------------- /ChangeLog.rst: -------------------------------------------------------------------------------- 1 | ================== 2 | PyBayes Change Log 3 | ================== 4 | 5 | This file mentions changes between PyBayes versions that are important for its users. Most 6 | recent versions are mentioned on top. 7 | 8 | Changes between 0.3 and 0.4 9 | =========================== 10 | 11 | * The optimised Cython PyBayes version was turned to use Cython memoryviews (with help 12 | from Ceygen_) instead of NumPy arrays where it makes sense. Cython memoryviews and NumPy 13 | arrays are mostly mutually compatible, but you may need to use :func:`np.asarray(...) 14 | ` or convert your code to use e.g. :func:`np.sum(pybayes_result) 15 | ` instead of :obj:`pybayes_result.sum() `. Alternatively, 16 | you can convert your code to use Cython memoryviews too. 17 | * Use of bundled Tokyo is replaced by the Ceygen_ project and Tokyo submodule is removed. 18 | * ParticleFilter.bayes() now ignores cond completely. Yell if you need it. 19 | * ParticleFilter lost last emp_pdf argument. Pass the same object as the init_pdf argument 20 | to achieve the same thing. 21 | * Test- and stress-suite no longer need PyBayes to be installed. (no privilege problems etc.) 22 | * Build-system was rewritten so that it is no longer an ugly hack. .pxd and .py files are now 23 | installed along .so (.dll) files for interoperability and additional openness. Better parsing of 24 | setup.py arguments and custom parameters visible in the --help command. 25 | * (C)Pdf shape() and cond_shape() functions are no longer abstract and just return 26 | `self.rv.dimension` and `self.cond_rv.dimension` respectively. CPdf subclasses therefore should 27 | not implement these methods. This is a backwards compatible change API-wise. 28 | 29 | .. _Ceygen: https://github.com/strohel/Ceygen 30 | -------------------------------------------------------------------------------- /HACKING.rst: -------------------------------------------------------------------------------- 1 | =================== 2 | PyBayes Development 3 | =================== 4 | 5 | This document should serve as a reminder to me and other possible PyBayes 6 | hackers about PyBayes coding style and conventions. 7 | 8 | General Layout and Principles 9 | ============================= 10 | 11 | PyBayes is developed with special dual-mode technique - it is both perfectly 12 | valid pure Python library and optimised cython-built binary python module. 13 | 14 | PyBayes modules are laid out with following rules: 15 | 16 | * all modules go directly into ``pybayes/.py`` (pure Python file) with 17 | cython augmentation file in ``pybayes/module.pxd`` 18 | * in future, bigger independent units can form subpackages 19 | * ``pybayes/wrappers/`` subpackage is special, it is the only package whose 20 | modules have different implementation for cython and for python. It is 21 | accomplished by .py (Python) and .pyx, .pxd (Cython) files. 22 | 23 | Tests and Stress Tests 24 | ====================== 25 | 26 | All methods of all PyBayes classes should have a unit test. Suppose you have 27 | a module ``pybayes/modname.py``, then unit tests for all classes in 28 | ``modname.py`` should go into ``pybayes/tests/test_modname.py``. You can also 29 | write stress test (something that runs considerably longer than a test and 30 | perhaps provides a simple benchmark) that would go into 31 | ``pybayes/tests/stress_modname.py``. 32 | 33 | Imports and cimports 34 | ==================== 35 | 36 | **No internal module** can ``import pybayes``! That would result in an infinite 37 | recursion. External PyBayes clients can and should, however, only ``import 38 | pybayes`` (and in future also ``import pybayes.subpackage``). From inside 39 | PyBayes just import relevant pybayes modules, e.g. ``import pdfs``. Notable 40 | exception from this rule is cimport, where (presumable due to a cython bug) 41 | ``from a.b cimport c`` sometimes doesn't work and one has to type ``from 42 | pybayes.a.b cimport c``. 43 | 44 | Imports in \*.py files should adhere to following rules: 45 | 46 | * import first system modules (sys, io..), then external modules (matplotlib..) 47 | and then pybayes modules. 48 | * **instead of** importing **numpy** directly use ``import wrappers._numpy as np``. 49 | This ensures that fast C alternatives are used in compiled mode. 50 | * **instead of** importing **numpy.linalg** directly use ``import wrappers._linalg as linalg``. 51 | * use ``import module [as abbrev]`` or, for commonly used symbols ``from module import symbol``. 52 | * ``from module import *`` shouldn't be used. 53 | 54 | Following rules apply to \*.pxd (cython augmentation) files: 55 | 56 | * no imports, just cimports. 57 | * use same import styles as in associated .py file. (``from module cimport`` vs. 58 | ``cimport module [as abbrev]``) 59 | * for numpy use ``cimport pybayes.wrappers._numpy as np`` 60 | * for numpy.linalg use ``cimport pybayes.wrappers._linalg as linalg`` 61 | 62 | *Above rules do not apply to* ``pybayes/tests``. *These modules are considered 63 | external and should behave as a client script.* 64 | 65 | Releasing PyBayes 66 | ================= 67 | 68 | Things to do when releasing new version (let it be **X.Y**) of PyBayes: 69 | 70 | Before Tagging 71 | -------------- 72 | 73 | 1. Set fallback version to **X.Y** in `setup.py` (around line 15) 74 | #. Set version to **X.Y** in `support/python-pybayes.spec` 75 | #. Ensure `ChangeLog.rst` mentions all important changes 76 | #. (Optional) update **short description** in `setup.py` **AND** `support/python-pybayes.spec` 77 | #. (Optional) update **long description** `README.rst` **AND** `support/python-pybayes.spec` 78 | 79 | Tagging 80 | ------- 81 | 82 | 1. Check everything, run tests and stresses for Python 2.7, 3.2 in both pure/Cython mode 83 | #. git tag -s **vX.Y** 84 | #. git-archive-all.sh --format tar --prefix PyBayes-**X.Y/** dist/PyBayes-**X.Y**.tar 85 | #. gzip dist/PyBayes-**X.Y**.tar 86 | #. ./setup.py register 87 | 88 | (do not use `./setup.py upload`, it does not work as some files are not in MANIFEST etc.) 89 | 90 | Publishing 91 | ---------- 92 | 93 | 1. Upload PyBayes-**X.Y**.tar.gz to https://github.com/strohel/PyBayes/downloads and 94 | http://pypi.python.org/pypi/PyBayes 95 | #. Build and upload docs: ``cd ../pybayes-doc && ./synchronize.sh`` 96 | #. Upload updated `python-pybayes.spec` file to 97 | https://build.opensuse.org/package/files?package=python-pybayes&project=home%3Astrohel 98 | #. If **short description** of PyBayes changed, update it manually at following places: 99 | 100 | * https://github.com/strohel/PyBayes 101 | #. If **long description** of PyBayes changed, update it manually at following places: 102 | 103 | * https://build.opensuse.org/package/show?package=python-pybayes&project=home%3Astrohel 104 | * http://scipy.org/Topical_Software 105 | * http://www.ohloh.net/p/pybayes 106 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Matěj Laitl 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | PyBayes 3 | ======= 4 | 5 | *Note: this software is not maintained since 2014.* 6 | 7 | About 8 | ===== 9 | 10 | PyBayes is an object-oriented Python library for recursive Bayesian 11 | estimation (Bayesian filtering) that is convenient to use. Already implemented are 12 | Kalman filter, particle filter and marginalized particle filter, all built atop of 13 | a light framework of probability density functions. PyBayes can optionally use Cython 14 | for large speed gains (Cython build can be several times faster in some situations). 15 | 16 | PyBayes is tested with Python 2.7, 3.2 and 3.3 (using 2to3). Future plans include 17 | more specialised variants of Kalman/particle filters and speed optimisations. 18 | 19 | PyBayes is being developed by Matěj Laitl, feel free to send me a mail to matej at laitl dot cz. 20 | See ChangeLog.rst file to review a list of most important changes in recent versions. 21 | 22 | Automatically generated **documentation** can be found at 23 | http://strohel.github.com/PyBayes-doc/ 24 | 25 | Licensing 26 | --------- 27 | 28 | PyBayes is distributed under the `MIT license`_. 29 | 30 | .. _`MIT license`: https://choosealicense.com/licenses/mit/ 31 | 32 | Obtaining PyBayes 33 | ================= 34 | 35 | PyBayes releases can be found in .tar.gz format at github_ or PyPI_. Binary packages for 36 | CentOS, Debian, Fedora, RHEL, OpenSUSE and Ubuntu can be downloaded from the 37 | `OpenSUSE Build Service`_; these packages are fast Cython builds. (with no requirement to 38 | install Cython for building) 39 | 40 | .. _github: https://github.com/strohel/PyBayes/downloads 41 | .. _PyPI: http://pypi.python.org/pypi/PyBayes 42 | .. _`OpenSUSE Build Service`: https://build.opensuse.org/package/show?package=python-pybayes&project=home%3Astrohel 43 | 44 | Development of PyBayes happens on http://github.com/strohel/PyBayes using git VCS 45 | and the most fresh development sources can be obtained using git:: 46 | 47 | # cd path/to/projects 48 | # git clone git://github.com/strohel/PyBayes.git 49 | 50 | Installing PyBayes 51 | ================== 52 | 53 | PyBayes uses standard Python distutils for building and installation. Follow 54 | these steps in order to install PyBayes: 55 | 56 | * download PyBayes, let's assume PyBayes-0.1.tar.gz filename 57 | * unpack it: 58 | 59 | ``tar -xvf PyBayes-0.1.tar.gz`` 60 | * change directory into PyBayes source: 61 | 62 | ``cd Pybayes-0.1`` 63 | * build and install (either run as root or install to a user-writeable 64 | directory [#alternate_install]_): 65 | 66 | ``./setup.py install`` 67 | 68 | .. [#alternate_install] http://docs.python.org/install/#alternate-installation 69 | 70 | **And you're done!** However, if you want PyBayes to be *considerably 71 | faster*, please read the following section. 72 | 73 | Advanced installation options 74 | ----------------------------- 75 | 76 | PyBayes can use Cython to build itself into binary Python module. Such binary modules are 77 | transparent to Python in a way that Python treats then as any other modules (you can 78 | ``import`` them as usual). Interpreter overhead is avoided and many other optimisation 79 | options arise this way. 80 | 81 | In order to build optimised PyBayes, you'll additionally need: 82 | 83 | * Cython_ Python to C compiler, version **0.18** or newer is recommended 84 | * working C compiler (GCC on Unix-like systems, MinGW or Microsoft Visual C on 85 | Windows [#install_cython]_) 86 | * NumPy_ numerical library for Python, version 1.5 or greater (NumPy is needed 87 | also in Python build, but older version suffice in that case) 88 | * Ceygen_ Python package 0.3 or greater installed to a standard location 89 | * On some Debian-based Linux distributions (Ubuntu) you'll need python-dev 90 | package that contains ``Python.h`` file that is needed by PyBayes 91 | 92 | .. _Cython: http://www.cython.org/ 93 | .. [#install_cython] http://docs.cython.org/src/quickstart/install.html 94 | .. _NumPy: http://numpy.scipy.org/ 95 | .. _Ceygen: https://github.com/strohel/Ceygen 96 | 97 | Proceed with following steps: 98 | 99 | 1. Install all required dependencies. They should be already available in your 100 | package manager if you use a modern Linux Distribution. 101 | 102 | #. Unpack and install PyBayes as described above, you should see following 103 | message during build: 104 | 105 | ``Cython and NumPy found, enabling optimised Cython build.`` 106 | 107 | * in order to be 100% sure that optimised build is used, you can add 108 | ``--use-cython=yes`` option to the ``./setup.py`` call. You can force pure 109 | Python mode even when Cython is installed, pass ``--use-cython=no``. By 110 | default, PyBayes auto-detects Cython and NumPy presence on system. 111 | * if you plan to profile code that uses optimised PyBayes, you may want to 112 | embed profiling information into PyBayes. This can be accomplished by 113 | passing ``--profile=yes`` to ``./setup.py``. The default is to omit 114 | profiling information in order to avoid performance penalties. 115 | * all standard and custom build parameters can be listed using ``./setup.py --help`` 116 | 117 | The best results performance-wise are achieved when also your code that uses or extends PyBayes is 118 | compiled by Cython and uses static typing where appropriate. Remember to 119 | ``cimport pybayes[.something]`` everytime you ``import pybayes[.something]`` so that fast Cython 120 | calling convention is used. 121 | 122 | Building Documentation 123 | ---------------------- 124 | 125 | *There is no need to build documentation yourself, an online version is at* 126 | http://strohel.github.com/PyBayes-doc/ 127 | 128 | PyBayes uses Sphinx_ to prepare documentation, version 1.0 or greater is required. 129 | The documentation is built separately from the python build process. 130 | In order to build it, change directory to `doc/` under PyBayes source directory 131 | (``cd [path_to_pybayes]/doc``) and issue ``make`` command. This will present you 132 | with a list of available documentation formats. To generate html documentation, 133 | for example, run ``make html`` and then point your browser to 134 | `[path_to_pybayes]/doc/_build/html/index.html`. 135 | 136 | PyBayes docs contain many mathematical expressions; Sphinx_ can use LaTeX_ to 137 | embed them as images into resulting HTML pages. Be sure to have LaTeX-enabled 138 | Sphinx if you want to see such nice things. 139 | 140 | .. _Sphinx: http://sphinx.pocoo.org/ 141 | .. _LaTeX: http://www.latex-project.org/ 142 | 143 | Testing 144 | ======= 145 | 146 | PyBayes comes with a comprehensive test and stress-suite that can and should be used to verify that 147 | your PyBayes build works as expected. 148 | 149 | Since version 0.4, testing is integrated into the `setup.py` script and can be run without 150 | installing PyBayes. In order to run PyBayes test-suite, simply issue ``./setup.py test`` from within 151 | the source directory. To run tests during installation procedure, simply install like this: 152 | ``./setup.py build test install``. With this command, failing tests prevent installation. 153 | 154 | If you want to test your already installed PyBayes instance, simply issue 155 | ``python -m pybayes.tests`` anytime, anywhere. :-) 156 | 157 | Stress-testing 158 | -------------- 159 | 160 | Stress-testing works similarly to unit testing since version 0.4, run it using ``./setup.py 161 | stress`` from the source directory. Already installed PyBayes can be stress-tested using 162 | ``python -m pybayes.stresses``. 163 | -------------------------------------------------------------------------------- /doc/ChangeLog.rst: -------------------------------------------------------------------------------- 1 | ../ChangeLog.rst -------------------------------------------------------------------------------- /doc/HACKING.rst: -------------------------------------------------------------------------------- 1 | ../HACKING.rst -------------------------------------------------------------------------------- /doc/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # Internal variables. 11 | PAPEROPT_a4 = -D latex_paper_size=a4 12 | PAPEROPT_letter = -D latex_paper_size=letter 13 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 14 | 15 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest 16 | 17 | help: 18 | @echo "Please use \`make ' where is one of" 19 | @echo " html to make standalone HTML files" 20 | @echo " dirhtml to make HTML files named index.html in directories" 21 | @echo " singlehtml to make a single large HTML file" 22 | @echo " pickle to make pickle files" 23 | @echo " json to make JSON files" 24 | @echo " htmlhelp to make HTML files and a HTML help project" 25 | @echo " qthelp to make HTML files and a qthelp project" 26 | @echo " devhelp to make HTML files and a Devhelp project" 27 | @echo " epub to make an epub" 28 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 29 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 30 | @echo " text to make text files" 31 | @echo " man to make manual pages" 32 | @echo " changes to make an overview of all changed/added/deprecated items" 33 | @echo " linkcheck to check all external links for integrity" 34 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 35 | 36 | clean: 37 | -rm -rf $(BUILDDIR)/* 38 | 39 | html: 40 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 41 | @echo 42 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 43 | 44 | dirhtml: 45 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 46 | @echo 47 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 48 | 49 | singlehtml: 50 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 51 | @echo 52 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 53 | 54 | pickle: 55 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 56 | @echo 57 | @echo "Build finished; now you can process the pickle files." 58 | 59 | json: 60 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 61 | @echo 62 | @echo "Build finished; now you can process the JSON files." 63 | 64 | htmlhelp: 65 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 66 | @echo 67 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 68 | ".hhp project file in $(BUILDDIR)/htmlhelp." 69 | 70 | qthelp: 71 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 72 | @echo 73 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 74 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 75 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/PyBayes.qhcp" 76 | @echo "To view the help file:" 77 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/PyBayes.qhc" 78 | 79 | devhelp: 80 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 81 | @echo 82 | @echo "Build finished." 83 | @echo "To view the help file:" 84 | @echo "# mkdir -p $$HOME/.local/share/devhelp/PyBayes" 85 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/PyBayes" 86 | @echo "# devhelp" 87 | 88 | epub: 89 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 90 | @echo 91 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 92 | 93 | latex: 94 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 95 | @echo 96 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 97 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 98 | "(use \`make latexpdf' here to do that automatically)." 99 | 100 | latexpdf: 101 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 102 | @echo "Running LaTeX files through pdflatex..." 103 | make -C $(BUILDDIR)/latex all-pdf 104 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 105 | 106 | text: 107 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 108 | @echo 109 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 110 | 111 | man: 112 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 113 | @echo 114 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 115 | 116 | changes: 117 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 118 | @echo 119 | @echo "The overview file is in $(BUILDDIR)/changes." 120 | 121 | linkcheck: 122 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 123 | @echo 124 | @echo "Link check complete; look for any errors in the above output " \ 125 | "or in $(BUILDDIR)/linkcheck/output.txt." 126 | 127 | doctest: 128 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 129 | @echo "Testing of doctests in the sources finished, look at the " \ 130 | "results in $(BUILDDIR)/doctest/output.txt." 131 | -------------------------------------------------------------------------------- /doc/README.rst: -------------------------------------------------------------------------------- 1 | ../README.rst -------------------------------------------------------------------------------- /doc/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # PyBayes documentation build configuration file, created by 4 | # sphinx-quickstart on Sun Dec 5 16:28:55 2010. 5 | # 6 | # This file is execfile()d with the current directory set to its containing dir. 7 | # 8 | # Note that not all possible configuration values are present in this 9 | # autogenerated file. 10 | # 11 | # All configuration values have a default; values that are commented out 12 | # serve to show the default. 13 | 14 | import subprocess, sys, os 15 | 16 | # If extensions (or modules to document with autodoc) are in another directory, 17 | # add these directories to sys.path here. If the directory is relative to the 18 | # documentation root, use os.path.abspath to make it absolute, like shown here. 19 | sys.path.insert(0, os.path.abspath('..')) 20 | 21 | # -- General configuration ----------------------------------------------------- 22 | 23 | # If your documentation needs a minimal Sphinx version, state it here. 24 | needs_sphinx = '1.0' 25 | 26 | # Add any Sphinx extension module names here, as strings. They can be extensions 27 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 28 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest', 'sphinx.ext.intersphinx', 'sphinx.ext.todo', 'sphinx.ext.coverage', 'sphinx.ext.pngmath', 'sphinx.ext.ifconfig', 'sphinx.ext.viewcode'] 29 | 30 | # Add any paths that contain templates here, relative to this directory. 31 | templates_path = ['_templates'] 32 | 33 | # The suffix of source filenames. 34 | source_suffix = '.rst' 35 | 36 | # The encoding of source files. 37 | #source_encoding = 'utf-8-sig' 38 | 39 | # The master toctree document. 40 | master_doc = 'index' 41 | 42 | # General information about the project. 43 | project = u'PyBayes' 44 | copyright = u'2010, Matěj Laitl' 45 | 46 | # The version info for the project you're documenting, acts as replacement for 47 | # |version| and |release|, also used in various other places throughout the 48 | # built documents. 49 | # 50 | # The short X.Y version. 51 | version = subprocess.check_output([sys.executable, os.path.join(os.path.dirname(os.path.abspath('.')), 'setup.py'), '--version']) 52 | # The full version, including alpha/beta/rc tags. 53 | release = version 54 | 55 | # The language for content autogenerated by Sphinx. Refer to documentation 56 | # for a list of supported languages. 57 | #language = None 58 | 59 | # There are two options for replacing |today|: either, you set today to some 60 | # non-false value, then it is used: 61 | #today = '' 62 | # Else, today_fmt is used as the format for a strftime call. 63 | #today_fmt = '%B %d, %Y' 64 | 65 | # List of patterns, relative to source directory, that match files and 66 | # directories to ignore when looking for source files. 67 | exclude_patterns = ['_build'] 68 | 69 | # The reST default role (used for this markup: `text`) to use for all documents. 70 | #default_role = None 71 | 72 | # If true, '()' will be appended to :func: etc. cross-reference text. 73 | #add_function_parentheses = True 74 | 75 | # If true, the current module name will be prepended to all description 76 | # unit titles (such as .. function::). 77 | #add_module_names = True 78 | 79 | # If true, sectionauthor and moduleauthor directives will be shown in the 80 | # output. They are ignored by default. 81 | #show_authors = False 82 | 83 | # The name of the Pygments (syntax highlighting) style to use. 84 | pygments_style = 'sphinx' 85 | 86 | # A list of ignored prefixes for module index sorting. 87 | #modindex_common_prefix = [] 88 | 89 | # Be nitpicky - warn about all undefined references 90 | nitpicky = True 91 | 92 | 93 | # -- Options for HTML output --------------------------------------------------- 94 | 95 | # The theme to use for HTML and HTML Help pages. See the documentation for 96 | # a list of builtin themes. 97 | html_theme = 'default' 98 | 99 | # Theme options are theme-specific and customize the look and feel of a theme 100 | # further. For a list of options available for each theme, see the 101 | # documentation. 102 | #html_theme_options = {} 103 | 104 | # Add any paths that contain custom themes here, relative to this directory. 105 | #html_theme_path = [] 106 | 107 | # The name for this set of Sphinx documents. If None, it defaults to 108 | # " v documentation". 109 | #html_title = None 110 | 111 | # A shorter title for the navigation bar. Default is the same as html_title. 112 | #html_short_title = None 113 | 114 | # The name of an image file (relative to this directory) to place at the top 115 | # of the sidebar. 116 | #html_logo = None 117 | 118 | # The name of an image file (within the static path) to use as favicon of the 119 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 120 | # pixels large. 121 | #html_favicon = None 122 | 123 | # Add any paths that contain custom static files (such as style sheets) here, 124 | # relative to this directory. They are copied after the builtin static files, 125 | # so a file named "default.css" will overwrite the builtin "default.css". 126 | html_static_path = ['_static'] 127 | 128 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 129 | # using the given strftime format. 130 | #html_last_updated_fmt = '%b %d, %Y' 131 | 132 | # If true, SmartyPants will be used to convert quotes and dashes to 133 | # typographically correct entities. 134 | #html_use_smartypants = True 135 | 136 | # Custom sidebar templates, maps document names to template names. 137 | #html_sidebars = {} 138 | 139 | # Additional templates that should be rendered to pages, maps page names to 140 | # template names. 141 | #html_additional_pages = {} 142 | 143 | # If false, no module index is generated. 144 | #html_domain_indices = True 145 | 146 | # If false, no index is generated. 147 | #html_use_index = True 148 | 149 | # If true, the index is split into individual pages for each letter. 150 | #html_split_index = False 151 | 152 | # If true, links to the reST sources are added to the pages. 153 | #html_show_sourcelink = True 154 | 155 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 156 | #html_show_sphinx = True 157 | 158 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 159 | #html_show_copyright = True 160 | 161 | # If true, an OpenSearch description file will be output, and all pages will 162 | # contain a tag referring to it. The value of this option must be the 163 | # base URL from which the finished HTML is served. 164 | #html_use_opensearch = '' 165 | 166 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 167 | #html_file_suffix = None 168 | 169 | # Output file base name for HTML help builder. 170 | htmlhelp_basename = 'PyBayesdoc' 171 | 172 | 173 | # -- Options for LaTeX output -------------------------------------------------- 174 | 175 | # Grouping the document tree into LaTeX files. List of tuples 176 | # (source start file, target name, title, author, documentclass [howto/manual]). 177 | latex_documents = [ 178 | ('index', # startdocname 179 | 'PyBayes.tex', # targetname 180 | u'PyBayes API Documentation', # title 181 | u'Matěj Laitl', # author 182 | 'manual', # documentclass: manual | howto 183 | True # toctree_only 184 | ), 185 | ] 186 | 187 | # The name of an image file (relative to this directory) to place at the top of 188 | # the title page. 189 | #latex_logo = None 190 | 191 | # For "manual" documents, if this is true, then toplevel headings are parts, 192 | # not chapters. 193 | #latex_use_parts = False 194 | 195 | # If true, show page references after internal links. 196 | latex_show_pagerefs = True 197 | 198 | # If true, show URL addresses after external links. 199 | latex_show_urls = False 200 | 201 | # Documents to append as an appendix to all manuals. 202 | #latex_appendices = [] 203 | 204 | # If false, no module index is generated. 205 | latex_domain_indices = False 206 | 207 | # A dict of additional options that override default .tex generation rules 208 | latex_elements = { 209 | 'papersize': 'a4paper,oneside', # a bit of hack with oneside 210 | 'pointsize': '11pt', 211 | 'fontpkg': '', # do not use Sphinx default 'times' fontpkg 212 | 'fncychap': '', # do not use Fancy Chapters 213 | 'preamble': '\\usepackage{amsfonts}\n', # so that \mathbb works 214 | } 215 | 216 | # -- Options for man output -------------------------------------------------- 217 | 218 | # One entry per manual page. List of tuples 219 | # (source start file, name, description, authors, manual section). 220 | man_pages = [ 221 | ('index', 'pybayes', u'PyBayes Documentation', 222 | [u'Matěj Laitl'], 1) 223 | ] 224 | 225 | 226 | # Example configuration for intersphinx: refer to the Python standard library. 227 | intersphinx_mapping = { 228 | 'http://docs.python.org/': None, 229 | 'http://docs.scipy.org/doc/numpy/': None, 230 | } 231 | 232 | 233 | # Options for autodoc 234 | autoclass_content = "class" # both, class 235 | autodoc_member_order = "bysource" 236 | autodoc_default_flags = ["members", "show_inheritance"] 237 | -------------------------------------------------------------------------------- /doc/filters.rst: -------------------------------------------------------------------------------- 1 | ================ 2 | Bayesian Filters 3 | ================ 4 | 5 | .. automodule:: pybayes.filters 6 | :no-members: 7 | 8 | Filter prototype 9 | ================ 10 | 11 | .. autoclass:: Filter 12 | 13 | Kalman Filter 14 | ============= 15 | 16 | .. autoclass:: KalmanFilter 17 | 18 | .. automethod:: __init__ 19 | 20 | Particle Filter Family 21 | ====================== 22 | 23 | .. autoclass:: ParticleFilter 24 | 25 | .. automethod:: __init__ 26 | 27 | .. autoclass:: MarginalizedParticleFilter 28 | 29 | .. automethod:: __init__ 30 | -------------------------------------------------------------------------------- /doc/index.rst: -------------------------------------------------------------------------------- 1 | ========================= 2 | PyBayes API Documentation 3 | ========================= 4 | 5 | .. automodule:: pybayes 6 | 7 | .. toctree:: 8 | :maxdepth: 2 9 | 10 | README 11 | ChangeLog 12 | pdfs 13 | filters 14 | HACKING 15 | 16 | Indices and tables 17 | ================== 18 | 19 | * :ref:`genindex` 20 | * :ref:`modindex` 21 | * :ref:`search` 22 | -------------------------------------------------------------------------------- /doc/pdfs.rst: -------------------------------------------------------------------------------- 1 | ============================= 2 | Probability Density Functions 3 | ============================= 4 | 5 | .. automodule:: pybayes.pdfs 6 | :no-members: 7 | 8 | Random Variables and their Components 9 | ===================================== 10 | 11 | .. autoclass:: RV 12 | 13 | .. automethod:: __init__ 14 | 15 | .. autoclass:: RVComp 16 | 17 | .. automethod:: __init__ 18 | 19 | Probability Density Function prototype 20 | ====================================== 21 | 22 | .. autoclass:: CPdf 23 | 24 | .. autoclass:: Pdf 25 | 26 | Unconditional Probability Density Functions (pdfs) 27 | ================================================== 28 | 29 | .. autoclass:: UniPdf 30 | 31 | .. automethod:: __init__ 32 | 33 | .. autoclass:: AbstractGaussPdf 34 | 35 | .. autoclass:: GaussPdf 36 | 37 | .. automethod:: __init__ 38 | 39 | .. autoclass:: LogNormPdf 40 | 41 | .. automethod:: __init__ 42 | 43 | .. autoclass:: TruncatedNormPdf 44 | 45 | .. automethod:: __init__ 46 | 47 | .. autoclass:: GammaPdf 48 | 49 | .. automethod:: __init__ 50 | 51 | .. autoclass:: InverseGammaPdf 52 | 53 | .. automethod:: __init__ 54 | 55 | .. autoclass:: AbstractEmpPdf 56 | 57 | .. autoclass:: EmpPdf 58 | 59 | .. automethod:: __init__ 60 | 61 | .. autoclass:: MarginalizedEmpPdf 62 | 63 | .. automethod:: __init__ 64 | 65 | .. autoclass:: ProdPdf 66 | 67 | .. automethod:: __init__ 68 | 69 | Conditional Probability Density Functions (cpdfs) 70 | ================================================= 71 | 72 | In this section, variable :math:`c` in math exressions denotes condition. 73 | 74 | .. autoclass:: MLinGaussCPdf 75 | 76 | .. automethod:: __init__ 77 | 78 | .. autoclass:: LinGaussCPdf 79 | 80 | .. automethod:: __init__ 81 | 82 | .. autoclass:: GaussCPdf 83 | 84 | .. automethod:: __init__ 85 | 86 | .. autoclass:: GammaCPdf 87 | 88 | .. automethod:: __init__ 89 | 90 | .. autoclass:: InverseGammaCPdf 91 | 92 | .. automethod:: __init__ 93 | 94 | .. autoclass:: ProdCPdf 95 | 96 | .. automethod:: __init__ 97 | -------------------------------------------------------------------------------- /pybayes/.gitignore: -------------------------------------------------------------------------------- 1 | *.c 2 | -------------------------------------------------------------------------------- /pybayes/__init__.pxd: -------------------------------------------------------------------------------- 1 | """ 2 | Cython definition file for PyBayes 3 | """ 4 | 5 | # TODO: cython bug(?): Cannot type from pdfs cimport Something 6 | from pybayes.pdfs cimport RVComp, RV, CPdf, Pdf, UniPdf, AbstractGaussPdf, GaussPdf, LogNormPdf, TruncatedNormPdf 7 | from pybayes.pdfs cimport GammaPdf, InverseGammaPdf, AbstractEmpPdf, EmpPdf, MarginalizedEmpPdf, ProdPdf 8 | from pybayes.pdfs cimport MLinGaussCPdf, LinGaussCPdf, GaussCPdf, GammaCPdf, InverseGammaCPdf, ProdCPdf 9 | from pybayes.filters cimport Filter, KalmanFilter, ParticleFilter, MarginalizedParticleFilter 10 | -------------------------------------------------------------------------------- /pybayes/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | PyBayes is an effort to create general, convenient to use and fast library for Bayesian filtering 3 | (and perhaps decision making in future). It is written in Python, but can make use of Cython to 4 | accelerate performance-critical code-paths. 5 | 6 | You may want to see `my bachelor thesis`_ for introduction to recursice Bayesian estimation, 7 | underlying software analysis and background information for PyBayes. 8 | 9 | .. _my bachelor thesis: thesis.pdf 10 | """ 11 | 12 | from .pdfs import RVComp, RV, CPdf, Pdf, UniPdf, AbstractGaussPdf, GaussPdf, LogNormPdf, TruncatedNormPdf 13 | from .pdfs import GammaPdf, InverseGammaPdf, AbstractEmpPdf, EmpPdf, MarginalizedEmpPdf, ProdPdf 14 | from .pdfs import MLinGaussCPdf, LinGaussCPdf, GaussCPdf, GammaCPdf, InverseGammaCPdf, ProdCPdf 15 | from .filters import Filter, KalmanFilter, ParticleFilter, MarginalizedParticleFilter 16 | -------------------------------------------------------------------------------- /pybayes/filters.pxd: -------------------------------------------------------------------------------- 1 | """Cython augmentation file for kalman.py""" 2 | 3 | cimport cython 4 | 5 | cimport pybayes.wrappers._linalg as linalg 6 | cimport pybayes.wrappers._numpy as np 7 | 8 | from pdfs cimport CPdf, Pdf, GaussPdf, EmpPdf, MarginalizedEmpPdf 9 | 10 | 11 | # workarounds for Cython: 12 | ctypedef double[:, :] double_2D 13 | ctypedef int[:] int_1D 14 | 15 | cdef class Filter(object): 16 | cpdef bint bayes(self, double[:] yt, double[:] cond = *) except False 17 | cpdef Pdf posterior(self) 18 | cpdef double evidence_log(self, double[:] yt) except? -1 19 | 20 | 21 | cdef class KalmanFilter(Filter): 22 | cdef readonly double[:, :] A, B, C, D, Q, R 23 | cdef readonly int n, k, j 24 | cdef readonly GaussPdf P, S 25 | 26 | @cython.locals(ret = KalmanFilter) 27 | cpdef KalmanFilter __copy__(self) 28 | 29 | @cython.locals(ret = KalmanFilter) 30 | cpdef KalmanFilter __deepcopy__(self, memo) 31 | 32 | @cython.locals(K = double_2D) 33 | cpdef bint bayes(self, double[:] yt, double[:] cond = *) except False 34 | 35 | 36 | cdef class ParticleFilter(Filter): 37 | cdef readonly CPdf p_xt_xtp, p_yt_xt 38 | cdef readonly EmpPdf emp 39 | 40 | cpdef bint bayes(self, double[:] yt, double[:] cond = *) except False 41 | 42 | 43 | cdef class MarginalizedParticleFilter(Filter): 44 | cdef readonly CPdf p_bt_btp 45 | cdef readonly KalmanFilter[:] kalmans 46 | cdef readonly MarginalizedEmpPdf memp 47 | 48 | @cython.locals(kalman = KalmanFilter) 49 | cpdef bint bayes(self, double[:] yt, double[:] cond = *) except False 50 | 51 | @cython.locals(indices = int_1D) 52 | cpdef bint _resample(self) except False 53 | -------------------------------------------------------------------------------- /pybayes/filters.py: -------------------------------------------------------------------------------- 1 | """ 2 | This module contains Bayesian filters. 3 | 4 | All classes from this module are currently imported to top-level pybayes module, 5 | so instead of ``from pybayes.filters import KalmanFilter`` you can type ``from 6 | pybayes import KalmanFilter``. 7 | """ 8 | 9 | from copy import deepcopy 10 | from math import exp 11 | 12 | from numpy import empty 13 | 14 | from .wrappers import _linalg as linalg 15 | from .wrappers import _numpy as np 16 | from .pdfs import CPdf, Pdf, GaussPdf, EmpPdf, MarginalizedEmpPdf 17 | 18 | 19 | class Filter(object): 20 | """Abstract prototype of a bayesian filter.""" 21 | 22 | def bayes(self, yt, cond = None): 23 | """Perform approximate or exact bayes rule. 24 | 25 | :param yt: observation at time t 26 | :type yt: 1D :class:`numpy.ndarray` 27 | :param cond: condition at time t. Exact meaning is defined by each filter 28 | :type cond: 1D :class:`numpy.ndarray` 29 | :return: always returns True (see :meth:`posterior` to get posterior density) 30 | """ 31 | raise NotImplementedError("Derived classes must implement this method") 32 | 33 | def posterior(self): 34 | """Return posterior probability density funcion (:class:`~pybayes.pdfs.Pdf`). 35 | 36 | :return: posterior density 37 | :rtype: :class:`~pybayes.pdfs.Pdf` 38 | 39 | *Filter implementations may decide to return a reference to their work pdf - it is not safe 40 | to modify it in any way, doing so may leave the filter in undefined state.* 41 | """ 42 | raise NotImplementedError("Derived classes must implement this method") 43 | 44 | def evidence_log(self, yt): 45 | """Return the logarithm of *evidence* function (also known as *marginal likelihood*) evaluated 46 | in point yt. 47 | 48 | :param yt: point which to evaluate the evidence in 49 | :type yt: :class:`numpy.ndarray` 50 | :rtype: double 51 | 52 | This is typically computed after :meth:`bayes` with the same observation: 53 | 54 | >>> filter.bayes(yt) 55 | >>> log_likelihood = filter.evidence_log(yt) 56 | """ 57 | raise NotImplementedError("Derived classes should implement this method, if feasible") 58 | 59 | 60 | class KalmanFilter(Filter): 61 | r"""Implementation of standard Kalman filter. **cond** in :meth:`bayes` is interpreted as 62 | control (intervention) input :math:`u_t` to the system. 63 | 64 | Kalman filter forms *optimal Bayesian solution* for the following system: 65 | 66 | .. math:: 67 | 68 | x_t &= A_t x_{t-1} + B_t u_t + v_{t-1} \quad \quad 69 | A_t \in \mathbb{R}^{n,n} \;\; 70 | B_t \in \mathbb{R}^{n,k} \;\; 71 | \;\; n \in \mathbb{N} 72 | \;\; k \in \mathbb{N}_0 \text{ (may be zero)} 73 | \\ 74 | y_t &= C_t x_t + D_t u_t + w_t \quad \quad \quad \;\; 75 | C_t \in \mathbb{R}^{j,n} \;\; 76 | D_t \in \mathbb{R}^{j,k} \;\; 77 | j \in \mathbb{N} \;\; j \leq n 78 | 79 | where :math:`x_t \in \mathbb{R}^n` is hidden state vector, :math:`y_t \in \mathbb{R}^j` is 80 | observation vector and :math:`u_t \in \mathbb{R}^k` is control vector. :math:`v_t` is normally 81 | distributed zero-mean process noise with covariance matrix :math:`Q_t`, :math:`w_t` is normally 82 | distributed zero-mean observation noise with covariance matrix :math:`R_t`. Additionally, intial 83 | pdf (**state_pdf**) has to be Gaussian. 84 | """ 85 | 86 | def __init__(self, A, B = None, C = None, D = None, Q = None, R = None, state_pdf = None): 87 | r"""Initialise Kalman filter. 88 | 89 | :param A: process model matrix :math:`A_t` from :class:`class description ` 90 | :type A: 2D :class:`numpy.ndarray` 91 | :param B: process control model matrix :math:`B_t` from :class:`class description 92 | `; may be None or unspecified for control-less systems 93 | :type B: 2D :class:`numpy.ndarray` 94 | :param C: observation model matrix :math:`C_t` from :class:`class description 95 | `; must be full-ranked 96 | :type C: 2D :class:`numpy.ndarray` 97 | :param D: observation control model matrix :math:`D_t` from :class:`class description 98 | `; may be None or unspecified for control-less systems 99 | :type D: 2D :class:`numpy.ndarray` 100 | :param Q: process noise covariance matrix :math:`Q_t` from :class:`class description 101 | `; must be positive definite 102 | :type Q: 2D :class:`numpy.ndarray` 103 | :param R: observation noise covariance matrix :math:`R_t` from :class:`class description 104 | `; must be positive definite 105 | :type R: 2D :class:`numpy.ndarray` 106 | :param state_pdf: initial state pdf; this object is referenced and used throughout whole 107 | life of KalmanFilter, so it is not safe to reuse state pdf for other purposes 108 | :type state_pdf: :class:`~pybayes.pdfs.GaussPdf` 109 | 110 | All matrices can be time-varying - you can modify or replace all above stated matrices 111 | providing that you don't change their shape and all constraints still hold. On the other 112 | hand, you **should not modify state_pdf** unless you really know what you are doing. 113 | 114 | >>> # initialise control-less Kalman filter: 115 | >>> kf = pb.KalmanFilter(A=np.array([[1.]]), 116 | C=np.array([[1.]]), 117 | Q=np.array([[0.7]]), R=np.array([[0.3]]), 118 | state_pdf=pb.GaussPdf(...)) 119 | """ 120 | 121 | # check type of pdf 122 | if not isinstance(state_pdf, GaussPdf): 123 | raise TypeError("state_pdf must be (a subclass of) GaussPdf") 124 | 125 | # check type of input arrays 126 | matrices = {"A":A, "B":B, "C":C, "D":D, "Q":Q, "R":R} 127 | for name in matrices: 128 | matrix = matrices[name] 129 | if name == 'B' and matrix is None: # we allow B to be unspecified 130 | continue 131 | if name == 'D' and matrix is None: # we allow D to be unspecified 132 | continue 133 | 134 | if matrix.ndim != 2: 135 | raise ValueError("{0} must have 2 dimensions (forming a matrix) {1} given".format( 136 | name, matrix.ndim)) 137 | 138 | # remember vector shapes 139 | self.n = state_pdf.shape() # dimension of state vector 140 | self.k = 0 if B is None else B.shape[1] # dimension of control vector 141 | self.j = C.shape[0] # dimension of observation vector 142 | 143 | # dict of required matrice shapes (sizes) 144 | shapes = { 145 | "A":(self.n, self.n), 146 | "B":(self.n, self.k), 147 | "C":(self.j, self.n), 148 | "D":(self.j, self.k), 149 | "Q":(self.n, self.n), 150 | "R":(self.j, self.j) 151 | } 152 | # check input matrix sizes 153 | for name in matrices: 154 | matrix = matrices[name] 155 | element_count = shapes[name][0] * shapes[name][1] 156 | if element_count == 0: 157 | assert(matrix is None) 158 | elif matrix.shape != shapes[name]: 159 | raise ValueError("Given shapes of state_pdf, B and C, matrix " + name + 160 | " must have shape " + str(shapes[name]) + ", " + 161 | str(matrix.shape) + " given") 162 | 163 | self.A, self.B, self.C, self.D, self.Q, self.R = A, B, C, D, Q, R 164 | 165 | self.P = state_pdf 166 | self.S = GaussPdf(np.array([0.]), np.array([[1.]])) # observation probability density function 167 | 168 | def __copy__(self): 169 | # type(self) is used because this method may be called for a derived class 170 | ret = type(self).__new__(type(self)) 171 | ret.A = self.A 172 | ret.B = self.B 173 | ret.C = self.C 174 | ret.D = self.D 175 | ret.Q = self.Q 176 | ret.R = self.R 177 | ret.n = self.n 178 | ret.k = self.k 179 | ret.j = self.j 180 | ret.P = self.P 181 | ret.S = self.S 182 | return ret 183 | 184 | def __deepcopy__(self, memo): 185 | # type(self) is used because this method may be called for a derived class 186 | ret = type(self).__new__(type(self)) 187 | # numeric arrays: 188 | ret.A = self.A.copy() 189 | ret.B = None if self.B is None else self.B.copy() 190 | ret.C = self.C.copy() 191 | ret.D = None if self.B is None else self.D.copy() 192 | ret.Q = self.Q.copy() 193 | ret.R = self.R.copy() 194 | ret.n = self.n # no need to copy integers 195 | ret.k = self.k 196 | ret.j = self.j 197 | ret.P = deepcopy(self.P, memo) # GaussPdfs: 198 | ret.S = deepcopy(self.S, memo) 199 | return ret 200 | 201 | def bayes(self, yt, cond = None): 202 | r"""Perform exact bayes rule. 203 | 204 | :param yt: observation at time t 205 | :type yt: 1D :class:`numpy.ndarray` 206 | :param cond: control (intervention) vector at time t. May be unspecified if the filter is 207 | control-less. 208 | :type cond: 1D :class:`numpy.ndarray` 209 | :return: always returns True (see :meth:`~Filter.posterior` to get posterior density) 210 | """ 211 | if yt.ndim != 1 or yt.shape[0] != self.j: 212 | raise ValueError("yt must have shape {0}. ({1} given)".format((self.j,), (yt.shape[0],))) 213 | if self.k > 0: 214 | if cond is None or cond.shape[0] != self.k: 215 | raise ValueError("cond must have shape {0}. ({1} given)".format((self.k,), (cond.shape[0],))) 216 | else: 217 | if cond is not None: 218 | raise ValueError("cond must be None as k == 0") 219 | 220 | # predict 221 | self.P.mu = np.dot_mv(self.A, self.P.mu) # prior state mean estimate 222 | if cond is not None: 223 | np.add_vv(self.P.mu, np.dot_mv(self.B, cond), self.P.mu) # self.P.mu += self.B * cond 224 | # prior state covariance estimate: 225 | self.P.R = np.dot_mm(np.dot_mm(self.A, self.P.R), self.A.T) # self.P.R = self.A * self.P.R * self.A' 226 | np.add_mm(self.P.R, self.Q, self.P.R) # self.P.R += self.Q 227 | 228 | # data update 229 | np.dot_mv(self.C, self.P.mu, self.S.mu) # prior observation mean estimate; self.S.mu = self.C * self.P.mu 230 | if cond is not None: 231 | np.add_vv(self.S.mu, np.dot_mv(self.D, cond), self.S.mu) # self.S.mu += self.D * cond 232 | # prior observation covariance estimate: 233 | np.dot_mm(np.dot_mm(self.C, self.P.R), self.C.T, self.S.R) # self.S.R = self.C * self.P.R * self.C' 234 | np.add_mm(self.S.R, self.R, self.S.R) # self.S.R += self.R 235 | 236 | # kalman gain 237 | K = np.dot_mm(np.dot_mm(self.P.R, self.C.T), linalg.inv(self.S.R)) 238 | 239 | # update according to observation 240 | # posterior state mean estimate: 241 | np.add_vv(self.P.mu, np.dot_mv(K, np.subtract_vv(yt, self.S.mu)), self.P.mu) # self.P.mu += K * (yt - self.S.mu) 242 | # posterior state covariance estimate: 243 | np.subtract_mm(self.P.R, np.dot_mm(np.dot_mm(K, self.C), self.P.R), self.P.R) # self.P.R -= K * self.C * self.P.R 244 | return True 245 | 246 | def posterior(self): 247 | return self.P 248 | 249 | def evidence_log(self, yt): 250 | return self.S.eval_log(yt) 251 | 252 | 253 | class ParticleFilter(Filter): 254 | r"""Standard particle filter implementation with resampling. 255 | 256 | Specifying proposal density is currently unsupported, but planned; speak up if you want it! 257 | Posterior pdf is represented using :class:`~pybayes.pdfs.EmpPdf` and takes following form: 258 | 259 | .. math:: p(x_t|y_{1:t}) = \sum_{i=1}^n \omega_i \delta ( x_t - x_t^{(i)} ) 260 | """ 261 | 262 | def __init__(self, n, init_pdf, p_xt_xtp, p_yt_xt): 263 | r"""Initialise particle filter. 264 | 265 | :param int n: number of particles 266 | :param init_pdf: either :class:`~pybayes.pdfs.EmpPdf` instance that will be used 267 | directly as a posterior (and should already have initial particles sampled) or 268 | any other probability density which initial particles are sampled from 269 | :type init_pdf: :class:`~pybayes.pdfs.Pdf` 270 | :param p_xt_xtp: :math:`p(x_t|x_{t-1})` cpdf of state in *t* given state in *t-1* 271 | :type p_xt_xtp: :class:`~pybayes.pdfs.CPdf` 272 | :param p_yt_xt: :math:`p(y_t|x_t)` cpdf of observation in *t* given state in *t* 273 | :type p_yt_xt: :class:`~pybayes.pdfs.CPdf` 274 | """ 275 | if not isinstance(n, int) or n < 1: 276 | raise TypeError("n must be a positive integer") 277 | if not isinstance(init_pdf, Pdf): 278 | raise TypeError("init_pdf must be an instance ot the Pdf class") 279 | if not isinstance(p_xt_xtp, CPdf) or not isinstance(p_yt_xt, CPdf): 280 | raise TypeError("both p_xt_xtp and p_yt_xt must be instances of the CPdf class") 281 | 282 | dim = init_pdf.shape() # dimension of state 283 | if p_xt_xtp.shape() != dim or p_xt_xtp.cond_shape() < dim: 284 | raise ValueError("Expected shape() and cond_shape() of p_xt_xtp will " 285 | + "be (respectively greater than) {0}; ({1}, {2}) given.".format(dim, 286 | p_xt_xtp.shape(), p_xt_xtp.cond_shape())) 287 | self.p_xt_xtp = p_xt_xtp 288 | if p_yt_xt.cond_shape() != dim: 289 | raise ValueError("Expected cond_shape() of p_yt_xt will be {0}; {1} given." 290 | .format(dim, p_yt_xt.cond_shape())) 291 | self.p_yt_xt = p_yt_xt 292 | 293 | if isinstance(init_pdf, EmpPdf): 294 | self.emp = init_pdf # use directly 295 | else: 296 | self.emp = EmpPdf(init_pdf.samples(n)) 297 | 298 | def bayes(self, yt, cond = None): 299 | r"""Perform Bayes rule for new measurement :math:`y_t`; *cond* is ignored. 300 | 301 | :param numpy.ndarray cond: optional condition that is passed to :math:`p(x_t|x_{t-1})` 302 | after :math:`x_{t-1}` so that is can be rewritten as: :math:`p(x_t|x_{t-1}, c)`. 303 | 304 | The algorithm is as follows: 305 | 306 | 1. generate new particles: :math:`x_t^{(i)} = \text{sample from } 307 | p(x_t^{(i)}|x_{t-1}^{(i)}) \quad \forall i` 308 | 2. recompute weights: :math:`\omega_i = p(y_t|x_t^{(i)}) 309 | \omega_i \quad \forall i` 310 | 3. normalise weights 311 | 4. resample particles 312 | """ 313 | for i in range(self.emp.particles.shape[0]): 314 | # generate new ith particle: 315 | self.emp.particles[i] = self.p_xt_xtp.sample(self.emp.particles[i]) 316 | 317 | # recompute ith weight: 318 | self.emp.weights[i] *= exp(self.p_yt_xt.eval_log(yt, self.emp.particles[i])) 319 | 320 | # assure that weights are normalised 321 | self.emp.normalise_weights() 322 | # resample 323 | self.emp.resample() 324 | return True 325 | 326 | def posterior(self): 327 | return self.emp 328 | 329 | 330 | class MarginalizedParticleFilter(Filter): 331 | r"""Simple marginalized particle filter implementation. Assume that tha state vector :math:`x` 332 | can be divided into two parts :math:`x_t = (a_t, b_t)` and that the pdf representing the process 333 | model can be factorised as follows: 334 | 335 | .. math:: p(x_t|x_{t-1}) = p(a_t|a_{t-1}, b_t) p(b_t | b_{t-1}) 336 | 337 | and that the :math:`a_t` part (given :math:`b_t`) can be estimated with (a subbclass of) the 338 | :class:`KalmanFilter`. Such system may be suitable for the marginalized particle filter, whose 339 | posterior pdf takes the form 340 | 341 | .. math:: 342 | 343 | p &= \sum_{i=1}^n \omega_i p(a_t | y_{1:t}, b_{1:t}^{(i)}) \delta(b_t - b_t^{(i)}) \\ 344 | p(a_t | y_{1:t}, b_{1:t}^{(i)}) &\text{ is posterior pdf of i}^{th} \text{ Kalman filter} \\ 345 | \text{where } \quad \quad \quad \quad \quad 346 | b_t^{(i)} &\text{ is value of the (b part of the) i}^{th} \text{ particle} \\ 347 | \omega_i \geq 0 &\text{ is weight of the i}^{th} \text{ particle} \quad \sum \omega_i = 1 348 | 349 | **Note:** currently :math:`b_t` is hard-coded to be process and observation noise covariance of the 350 | :math:`a_t` part. This will be changed soon and :math:`b_t` will be passed as condition to 351 | :meth:`KalmanFilter.bayes`. 352 | """ 353 | 354 | def __init__(self, n, init_pdf, p_bt_btp, kalman_args, kalman_class = KalmanFilter): 355 | r"""Initialise marginalized particle filter. 356 | 357 | :param int n: number of particles 358 | :param init_pdf: probability density which initial particles are sampled from. (both 359 | :math:`a_t` and :math:`b_t` parts) 360 | :type init_pdf: :class:`~pybayes.pdfs.Pdf` 361 | :param p_bt_btp: :math:`p(b_t|b_{t-1})` cpdf of the (b part of the) state in *t* given 362 | state in *t-1* 363 | :type p_bt_btp: :class:`~pybayes.pdfs.CPdf` 364 | :param dict kalman_args: arguments for the Kalman filter, passed as dictionary; *state_pdf* 365 | key should not be speficied as it is supplied by the marginalized particle filter 366 | :param class kalman_class: class of the filter used for the :math:`a_t` part of the system; 367 | defaults to :class:`KalmanFilter` 368 | """ 369 | if not isinstance(n, int) or n < 1: 370 | raise TypeError("n must be a positive integer") 371 | if not isinstance(init_pdf, Pdf) or not isinstance(p_bt_btp, CPdf): 372 | raise TypeError("init_pdf must be a Pdf and p_bt_btp must be a CPdf") 373 | if not issubclass(kalman_class, KalmanFilter): 374 | raise TypeError("kalman_class must be a subclass (not an instance) of KalmanFilter") 375 | b_shape = p_bt_btp.shape() 376 | if p_bt_btp.cond_shape() != b_shape: 377 | raise ValueError("p_bt_btp's shape ({0}) and cond shape ({1}) must both be {2}".format( 378 | p_bt_btp.shape(), p_bt_btp.cond_shape(), b_shape)) 379 | self.p_bt_btp = p_bt_btp 380 | a_shape = init_pdf.shape() - b_shape 381 | 382 | # this will be removed when hardcoding Q,R into kalman filter will be removed 383 | kalman_args['Q'] = np.array([[-123.]]) 384 | kalman_args['R'] = np.array([[-494658.]]) 385 | 386 | # generate both initial parts of particles 387 | init_particles = init_pdf.samples(n) 388 | 389 | # create all Kalman filters first 390 | self.kalmans = empty(n, dtype=KalmanFilter) # array of references to Kalman filters 391 | gausses = empty(n, dtype=GaussPdf) # array of Kalman filter state pdfs 392 | for i in range(n): 393 | gausses[i] = GaussPdf(init_particles[i,0:a_shape], np.array([[1.]])) 394 | kalman_args['state_pdf'] = gausses[i] 395 | self.kalmans[i] = kalman_class(**kalman_args) 396 | # construct apost pdf. Important: reference to ith GaussPdf is shared between ith Kalman 397 | # filter's state_pdf and ith memp't gauss 398 | self.memp = MarginalizedEmpPdf(gausses, init_particles[:,a_shape:]) 399 | 400 | def __str__(self): 401 | ret = "" 402 | for i in range(self.kalmans.shape[0]): 403 | ret += " {0}: {1:0<5.3f} * {2} {3} kf.S: {4}\n".format(i, self.memp.weights[i], 404 | self.memp.gausses[i], self.memp.particles[i], self.kalmans[i].S) 405 | return ret[:-1] # trim the last newline 406 | 407 | def bayes(self, yt, cond = None): 408 | r"""Perform Bayes rule for new measurement :math:`y_t`. Uses following algorithm: 409 | 410 | 1. generate new b parts of particles: :math:`b_t^{(i)} = \text{sample from } 411 | p(b_t^{(i)}|b_{t-1}^{(i)}) \quad \forall i` 412 | 2. :math:`\text{set } Q_i = b_t^{(i)} \quad R_i = b_t^{(i)}` where :math:`Q_i, R_i` is 413 | covariance of process (respectively observation) noise in ith Kalman filter. 414 | 3. perform Bayes rule for each Kalman filter using passed observation :math:`y_t` 415 | 4. recompute weights: :math:`\omega_i = p(y_t | y_{1:t-1}, b_t^{(i)}) \omega_i` where 416 | :math:`p(y_t | y_{1:t-1}, b_t^{(i)})` is *evidence* (*marginal likelihood*) pdf of ith Kalman 417 | filter. 418 | 5. normalise weights 419 | 6. resample particles 420 | """ 421 | for i in range(self.kalmans.shape[0]): 422 | # generate new b_t 423 | self.memp.particles[i] = self.p_bt_btp.sample(self.memp.particles[i]) 424 | 425 | # assign b_t to kalman filter 426 | # TODO: more general and correct apprach would be some kind of QRKalmanFilter that would 427 | # accept b_t in condition. This is planned in future. 428 | kalman = self.kalmans[i] 429 | kalman.Q[0,0] = self.memp.particles[i,0] 430 | kalman.R[0,0] = self.memp.particles[i,0] 431 | 432 | kalman.bayes(yt) 433 | 434 | self.memp.weights[i] *= exp(kalman.evidence_log(yt)) 435 | 436 | # make sure that weights are normalised 437 | self.memp.normalise_weights() 438 | # resample particles 439 | self._resample() 440 | return True 441 | 442 | def _resample(self): 443 | indices = self.memp.get_resample_indices() 444 | np.reindex_vv(self.kalmans, indices) # resample kalman filters (makes references, not hard copies) 445 | np.reindex_mv(self.memp.particles, indices) # resample particles 446 | for i in range(self.kalmans.shape[0]): 447 | if indices[i] == i: # copy only when needed 448 | continue 449 | self.kalmans[i] = deepcopy(self.kalmans[i]) # we need to deep copy ith kalman 450 | self.memp.gausses[i] = self.kalmans[i].P # reassign reference to correct (new) state pdf 451 | 452 | self.memp.weights[:] = 1./self.kalmans.shape[0] # set weights to 1/n 453 | return True 454 | 455 | def posterior(self): 456 | return self.memp 457 | -------------------------------------------------------------------------------- /pybayes/pdfs.pxd: -------------------------------------------------------------------------------- 1 | """Cython augmentation file for pdfs.py""" 2 | 3 | cimport cython 4 | 5 | cimport pybayes.wrappers._linalg as linalg 6 | cimport pybayes.wrappers._numpy as np 7 | 8 | 9 | # workarounds for Cython: 10 | ctypedef double[:] double_1D 11 | ctypedef double[:, :] double_2D 12 | ctypedef int[:] int_1D 13 | 14 | cdef class RVComp(object): 15 | cdef readonly int dimension 16 | cdef public str name 17 | 18 | 19 | cdef class RV(object): 20 | cdef readonly int dimension 21 | cdef public str name 22 | cdef readonly list components 23 | 24 | @cython.locals(ret = RV) 25 | cpdef RV __copy__(self) 26 | 27 | @cython.locals(ret = RV) 28 | cpdef RV __deepcopy__(self, memo) 29 | 30 | cpdef bint contains(self, RVComp component) except? False 31 | cpdef bint contains_all(self, test_components) except? False 32 | cpdef bint contains_any(self, test_components) except? False 33 | 34 | cdef bint _add_component(self, RVComp component) except False 35 | 36 | 37 | cdef class CPdf(object): 38 | cdef public RV rv, cond_rv 39 | 40 | cpdef int shape(self) except -1 41 | cpdef int cond_shape(self) except -1 42 | cpdef double[:] mean(self, double[:] cond = *) 43 | cpdef double[:] variance(self, double[:] cond = *) 44 | cpdef double eval_log(self, double[:] x, double[:] cond = *) except? -1 45 | cpdef double[:] sample(self, double[:] cond = *) 46 | @cython.locals(ret = double_2D) 47 | cpdef double[:, :] samples(self, int n, double[:] cond = *) 48 | 49 | cpdef bint _check_cond(self, double[:] cond) except False 50 | cpdef bint _check_x(self, double[:] x) except False 51 | cpdef bint _set_rvs(self, int exp_shape, rv, int exp_cond_shape, cond_rv) except False 52 | 53 | 54 | cdef class Pdf(CPdf): 55 | cpdef bint _set_rv(self, int exp_shape, rv) except False 56 | 57 | 58 | cdef class UniPdf(Pdf): 59 | # intentionally ndarrays 60 | cdef public np.ndarray a, b 61 | 62 | 63 | cdef class AbstractGaussPdf(Pdf): 64 | cdef public double[:] mu 65 | cdef public double[:, :] R 66 | 67 | @cython.locals(ret = AbstractGaussPdf) 68 | cpdef AbstractGaussPdf __copy__(self) 69 | 70 | @cython.locals(ret = AbstractGaussPdf) 71 | cpdef AbstractGaussPdf __deepcopy__(self, memo) 72 | 73 | 74 | cdef class GaussPdf(AbstractGaussPdf): 75 | 76 | @cython.locals(log_norm = double, distance = double_1D, log_val = double) 77 | cpdef double eval_log(self, double[:] x, double[:] cond = *) except? -1 78 | 79 | @cython.locals(z = double_1D) 80 | cpdef double[:] sample(self, double[:] cond = *) 81 | 82 | 83 | cdef class LogNormPdf(AbstractGaussPdf): 84 | pass # everything inherited from AbstractGaussPdf 85 | 86 | 87 | cdef class TruncatedNormPdf(Pdf): 88 | cdef public double mu, sigma_sq, a, b 89 | 90 | cdef double _pdf(self, double x) 91 | cdef double _cdf(self, double x) 92 | 93 | cdef class GammaPdf(Pdf): 94 | cdef public double k, theta 95 | 96 | 97 | cdef class InverseGammaPdf(Pdf): 98 | cdef public double alpha, beta 99 | 100 | 101 | cdef class AbstractEmpPdf(Pdf): 102 | cdef public double[:] weights 103 | cdef public double[:, :] particles 104 | 105 | @cython.locals(wsum = double) 106 | cpdef bint normalise_weights(self) except False 107 | 108 | @cython.locals(n = int, cum_weights = double_1D, u = double_1D, baby_indices = int_1D, j = int) 109 | cpdef int[:] get_resample_indices(self) 110 | 111 | 112 | cdef class EmpPdf(AbstractEmpPdf): 113 | cpdef bint resample(self) except False 114 | 115 | 116 | cdef class MarginalizedEmpPdf(AbstractEmpPdf): 117 | cdef public GaussPdf[:] gausses 118 | cdef public int _gauss_shape, _part_shape 119 | 120 | @cython.locals(gauss = GaussPdf) 121 | cpdef double[:] mean(self, double[:] cond = *) 122 | 123 | @cython.locals(nom2 = double_1D, temp = double_1D, gauss = GaussPdf, mean = double_1D, var = double_1D) 124 | cpdef double[:] variance(self, double[:] cond = *) 125 | 126 | 127 | cdef class ProdPdf(Pdf): 128 | cdef readonly Pdf[:] factors 129 | cdef readonly int[:] shapes 130 | cdef int _shape 131 | 132 | @cython.locals(curr = int, factor = Pdf, i = int, ret = double_1D) 133 | cpdef double[:] mean(self, double[:] cond = *) 134 | 135 | @cython.locals(curr = int, factor = Pdf, i = int, ret = double_1D) 136 | cpdef double[:] variance(self, double[:] cond = *) 137 | 138 | @cython.locals(curr = int, factor = Pdf, i = int, ret = double) 139 | cpdef double eval_log(self, double[:] x, double[:] cond = *) 140 | 141 | @cython.locals(curr = int, factor = Pdf, i = int, ret = double_1D) 142 | cpdef double[:] sample(self, double[:] cond = *) 143 | 144 | cpdef int _calculate_shape(self) except -1 145 | 146 | 147 | cdef class MLinGaussCPdf(CPdf): 148 | cdef public double[:, :] A 149 | cdef public double[:] b 150 | cdef AbstractGaussPdf gauss 151 | 152 | cdef bint _set_mean(self, double[:] cond) except False 153 | 154 | 155 | cdef class LinGaussCPdf(CPdf): 156 | cdef public double a, b, c, d 157 | cdef AbstractGaussPdf gauss 158 | 159 | @cython.locals(c0 = double, c1 = double) 160 | cdef bint _set_gauss_params(self, double[:] cond) except False 161 | 162 | 163 | cdef class GaussCPdf(CPdf): 164 | cdef int _shape, _cond_shape 165 | cdef public object f, g # callables 166 | cdef AbstractGaussPdf gauss 167 | 168 | cdef bint _set_gauss_params(self, double[:] cond) except False 169 | 170 | 171 | cdef class GammaCPdf(CPdf): 172 | cdef public double gamma 173 | cdef public GammaPdf gamma_pdf 174 | 175 | 176 | cdef class InverseGammaCPdf(CPdf): 177 | cdef public double gamma 178 | cdef public InverseGammaPdf igamma_pdf 179 | 180 | 181 | cdef class ProdCPdf(CPdf): 182 | cdef readonly CPdf[:] factors 183 | cdef readonly list in_indeces, out_indeces # dtype=array of ints 184 | cdef int _shape, _cond_shape 185 | -------------------------------------------------------------------------------- /pybayes/stresses/__init__.py: -------------------------------------------------------------------------------- 1 | """PyBayes' stress tests""" 2 | 3 | from pybayes.stresses.stress_filters import * 4 | -------------------------------------------------------------------------------- /pybayes/stresses/__main__.py: -------------------------------------------------------------------------------- 1 | """PyBayes stress-suite runner. Used when user calls `python -m pybayes.stresses""" 2 | 3 | import unittest as ut 4 | 5 | from pybayes.stresses import * 6 | 7 | 8 | if __name__ == '__main__': 9 | ut.main() 10 | -------------------------------------------------------------------------------- /pybayes/stresses/data/stress_kalman_data.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/pybayes/stresses/data/stress_kalman_data.mat -------------------------------------------------------------------------------- /pybayes/stresses/stress_filters.pxd: -------------------------------------------------------------------------------- 1 | cimport cython 2 | cimport numpy as np 3 | 4 | cimport pybayes as pb 5 | 6 | 7 | #@cython.boundscheck(False) 8 | #@cython.wraparound(False) 9 | @cython.locals(kalman = pb.KalmanFilter, 10 | y = np.ndarray, 11 | u = np.ndarray, 12 | mean = np.ndarray, 13 | t = int, 14 | N = int) 15 | cpdef run_kalman_on_mat_data(input_file, output_file, timer) 16 | 17 | @cython.locals(nr_steps = int, 18 | pf = pb.Filter) 19 | cpdef run_pf(timer, pf_opts, nr_particles, pf_class) 20 | -------------------------------------------------------------------------------- /pybayes/stresses/stress_filters.py: -------------------------------------------------------------------------------- 1 | """Stresses for kalman filters""" 2 | 3 | import numpy as np 4 | try: 5 | import matplotlib.pyplot as plt 6 | except ImportError: 7 | plt = None 8 | try: 9 | import scipy.io 10 | except ImportError: 11 | scipy = None 12 | 13 | from os.path import dirname, join 14 | import unittest as ut 15 | 16 | import pybayes as pb 17 | from support import timed 18 | 19 | 20 | def run_kalman_on_mat_data(input_file, output_file, timer): 21 | # this should be here so that only this stress fails when scipy is not installed 22 | loadmat = scipy.io.loadmat 23 | savemat = scipy.io.savemat 24 | 25 | d = loadmat(input_file, struct_as_record=True, mat_dtype=True) 26 | 27 | mu0 = np.reshape(d.pop('mu0'), (-1,)) # otherwise we would get 2D array of shape (1xN) 28 | P0 = d.pop('P0') 29 | y = d.pop('y').T 30 | u = d.pop('u').T 31 | #x = d.pop('x').T 32 | x = None 33 | 34 | gauss = pb.GaussPdf(mu0, P0) 35 | kalman = pb.KalmanFilter(d['A'], d['B'], d['C'], d['D'], d['Q'], d['R'], gauss) 36 | 37 | N = y.shape[0] 38 | n = mu0.shape[0] 39 | mean = np.zeros((N, n)) 40 | var = np.zeros((N, n)) 41 | 42 | timer.start() 43 | for t in xrange(1, N): # the 1 start offset is intentional 44 | kalman.bayes(y[t], u[t]) 45 | mean[t] = kalman.posterior().mean() 46 | #var[t] = kalman.posterior().variance() 47 | timer.stop() 48 | 49 | var = np.sqrt(var) # to get standard deviation 50 | plt = None # turn off plotting for now 51 | if plt: 52 | axis = np.arange(N) 53 | plt.plot(axis, x[:,0], 'k-', label='x_1') 54 | plt.plot(axis, x[:,1], 'k--', label='x_2') 55 | plt.errorbar(axis, mean[:,0], fmt='s', label='mu_1') # yerror=var[:,0] 56 | plt.errorbar(axis, mean[:,1], fmt='D', label='mu_2') # yerror=var[:,1] 57 | plt.legend() 58 | plt.show() 59 | 60 | savemat(output_file, {"Mu_py":mean.T, "exec_time_pybayes":timer.spent[0]}, oned_as='row') 61 | 62 | 63 | class PfOptionsA(object): 64 | """Class that represents options for a particle filter""" 65 | 66 | def __init__(self, nr_steps): 67 | print "Generating random data for particle filter stresses A..." 68 | self.nr_steps = nr_steps 69 | 70 | # prepare random variable components: 71 | a_t, b_t = pb.RVComp(1, 'a_t'), pb.RVComp(1, 'b_t') # state in t 72 | a_tp, b_tp = pb.RVComp(1, 'a_{t-1}'), pb.RVComp(1, 'b_{t-1}') # state in t-1 73 | 74 | # arguments to Kalman filter part of the marginalized particle filter 75 | self.kalman_args = {} 76 | 77 | # prepare callback functions 78 | sigma_sq = np.array([0.0001]) 79 | def f(cond): # log(b_{t-1}) - 1/2 \sigma^2 80 | return np.log(cond) - sigma_sq/2. 81 | def g(cond): # \sigma^2 82 | return sigma_sq 83 | 84 | # p(a_t | a_{t-1} b_t) density: 85 | p_at_atpbt = pb.LinGaussCPdf(1., 0., 1., 0., [a_t], [a_tp, b_t]) 86 | self.kalman_args['A'] = np.array([[1.]]) # process model 87 | # p(b_t | b_{t-1}) density: 88 | self.p_bt_btp = pb.GaussCPdf(1, 1, f, g, rv=[b_t], cond_rv=[b_tp], base_class=pb.LogNormPdf) 89 | # p(x_t | x_{t-1}) density: 90 | self.p_xt_xtp = pb.ProdCPdf((p_at_atpbt, self.p_bt_btp), [a_t, b_t], [a_tp, b_tp]) 91 | 92 | # prepare p(y_t | x_t) density: 93 | self.p_yt_xt = pb.LinGaussCPdf(1., 0., 1., 0.) 94 | self.kalman_args['C'] = np.array([[1.]]) # observation model 95 | 96 | # Initial [a_t, b_t] from .. to: 97 | self.init_range = np.array([[-18., 0.3], [-14., 0.7]]) 98 | init_mean = (self.init_range[0] + self.init_range[1])/2. 99 | 100 | x_t = np.zeros((nr_steps, 2)) 101 | x_t[0] = init_mean.copy() 102 | y_t = np.empty((nr_steps, 1)) 103 | for i in range(nr_steps): 104 | # set b_t: 105 | x_t[i,1] = i/100. + init_mean[1] 106 | # simulate random process: 107 | x_t[i,0:1] = p_at_atpbt.sample(x_t[i]) # this is effectively [a_{t-1}, b_t] 108 | y_t[i] = self.p_yt_xt.sample(x_t[i]) 109 | # DEBUG: print "simulated x_{0} = {1}".format(i, x_t[i]) 110 | # DEBUG: print "simulated y_{0} = {1}".format(i, y_t[i]) 111 | self.x_t = x_t 112 | self.y_t = y_t 113 | 114 | 115 | def run_pf(timer, pf_opts, nr_particles, pf_class): 116 | nr_steps = pf_opts.nr_steps # number of time steps 117 | 118 | # prepare initial particle density: 119 | init_pdf = pb.UniPdf(pf_opts.init_range[0], pf_opts.init_range[1]) 120 | 121 | # construct particle filter 122 | if pf_class is pb.ParticleFilter: 123 | pf = pf_class(nr_particles, init_pdf, pf_opts.p_xt_xtp, pf_opts.p_yt_xt) 124 | elif pf_class is pb.MarginalizedParticleFilter: 125 | pf = pf_class(nr_particles, init_pdf, pf_opts.p_bt_btp, pf_opts.kalman_args) 126 | else: 127 | raise NotImplementedError("This switch case not handled") 128 | 129 | x_t = pf_opts.x_t 130 | y_t = pf_opts.y_t 131 | mean = np.empty((nr_steps, 2)) 132 | 133 | timer.start() 134 | for i in range(nr_steps): 135 | pf.bayes(y_t[i]) 136 | mean[i] = pf.posterior().mean() 137 | timer.stop() 138 | cumerror = np.sum((mean - x_t)**2, 0) 139 | print " {0}-{3} cummulative error for {1} steps: {2}".format( 140 | nr_particles, nr_steps, np.sqrt(cumerror), pf_class.__name__) 141 | plt = None # disable plotting for now 142 | if plt: 143 | x = np.arange(nr_steps) 144 | plt.plot(x, mean[:,0], 'x', label="{0}: {1}".format(nr_particles, pf_class.__name__)) 145 | plt.plot(x, mean[:,1], '+', label="{0}: {1}".format(nr_particles, pf_class.__name__)) 146 | if plt and nr_particles == 90 and pf_class == pb.MarginalizedParticleFilter: 147 | plt.plot(x, x_t[:,0], '-') 148 | plt.plot(x, x_t[:,1], '--') 149 | plt.legend() 150 | plt.show() 151 | 152 | 153 | class StressFilters(ut.TestCase): 154 | pf_nr_steps = 100 # number of steps for particle filter 155 | pf_opts_a = PfOptionsA(pf_nr_steps) 156 | 157 | @ut.skipUnless(scipy, "Kalman stress needs SciPy installed") 158 | @timed 159 | def test_kalman(self, timer): 160 | input_file = join(dirname(__file__), "data", "stress_kalman_data.mat") 161 | output_file = "stress_kalman_res.mat" 162 | run_kalman_on_mat_data(input_file, output_file, timer) 163 | 164 | @timed 165 | def test_pf_a_1(self, timer): 166 | run_pf(timer, self.pf_opts_a, 10, pb.ParticleFilter) 167 | 168 | @timed 169 | def test_pf_a_1_marg(self, timer): 170 | run_pf(timer, self.pf_opts_a, 10, pb.MarginalizedParticleFilter) 171 | 172 | @timed 173 | def test_pf_a_2(self, timer): 174 | run_pf(timer, self.pf_opts_a, 30, pb.ParticleFilter) 175 | 176 | @timed 177 | def test_pf_a_2_marg(self, timer): 178 | run_pf(timer, self.pf_opts_a, 30, pb.MarginalizedParticleFilter) 179 | 180 | @timed 181 | def test_pf_a_3(self, timer): 182 | run_pf(timer, self.pf_opts_a, 90, pb.ParticleFilter) 183 | 184 | @timed 185 | def test_pf_a_3_marg(self, timer): 186 | run_pf(timer, self.pf_opts_a, 90, pb.MarginalizedParticleFilter) 187 | -------------------------------------------------------------------------------- /pybayes/stresses/support.py: -------------------------------------------------------------------------------- 1 | """Various support methods for stresses""" 2 | 3 | import numpy as np 4 | 5 | import functools 6 | import time 7 | 8 | 9 | class Timer(object): 10 | """Simple timer used to measure real and cpu time of stresses.""" 11 | 12 | def __init__(self): 13 | self.spent = "undefined", "undefined" 14 | 15 | def start(self): 16 | self.start_time = np.array([time.time(), time.clock()]) 17 | 18 | def stop(self): 19 | self.spent = np.array([time.time(), time.clock()]) - self.start_time 20 | 21 | def __str__(self): 22 | return "Time spent: {0}s real time; {1}s CPU time".format(self.spent[0], 23 | self.spent[1]) 24 | 25 | 26 | def timed(func): 27 | """Decorator to mark a test as timed, provides timer argument""" 28 | @functools.wraps(func) 29 | def wrapper(self): 30 | timer = Timer() 31 | func(self, timer) 32 | print "{0}(): {1}".format(func.__name__, timer) 33 | return wrapper 34 | -------------------------------------------------------------------------------- /pybayes/tests/__init__.py: -------------------------------------------------------------------------------- 1 | """PyBayes' tests""" 2 | 3 | from pybayes.tests.test_filters import * 4 | from pybayes.tests.test_wrappers_linalg import * 5 | from pybayes.tests.test_wrappers_numpy import * 6 | from pybayes.tests.test_pdfs import * 7 | -------------------------------------------------------------------------------- /pybayes/tests/__main__.py: -------------------------------------------------------------------------------- 1 | """PyBayes test-suite runner. Used when user calls `python -m pybayes.tests""" 2 | 3 | import unittest as ut 4 | 5 | from pybayes.tests import * 6 | 7 | 8 | if __name__ == '__main__': 9 | ut.main() 10 | -------------------------------------------------------------------------------- /pybayes/tests/support.py: -------------------------------------------------------------------------------- 1 | """Various support methods for tests""" 2 | 3 | import functools 4 | import numpy as np 5 | import sys 6 | import unittest as ut 7 | try: 8 | from unittest.case import _ExpectedFailure as ExpectedFailure 9 | except ImportError: 10 | ExpectedFailure = None 11 | 12 | 13 | def stochastic(func): 14 | """Decorator to mark test as stochastic - these tests are not fatal when they fail.""" 15 | @functools.wraps(func) 16 | def wrapper(*args, **kwargs): 17 | try: 18 | return func(*args, **kwargs) 19 | except ut.TestCase.failureException: 20 | if ExpectedFailure is not None: # added in Py 2.7 21 | raise ExpectedFailure(sys.exc_info()) 22 | wrapper.__doc__ += ' (stochastic, failures ignored)' 23 | return wrapper 24 | 25 | 26 | class PbTestCase(ut.TestCase): 27 | """Test case that adds some numeric assert functions""" 28 | 29 | def assertApproxEqual(self, X, Y): 30 | """Return true if X = Y to within machine precision 31 | 32 | Function for checking that different matrices from different 33 | computations are in some sense "equal" in the verification tests. 34 | """ 35 | X = np.asarray(X) 36 | Y = np.asarray(Y) 37 | fuzz = 1.0e-8 38 | 39 | self.assertEqual(X.ndim, Y.ndim) 40 | self.assertEqual(X.shape, Y.shape) 41 | 42 | if np.all(X == Y): # catches -inf == -inf etc. 43 | return 44 | if np.all(abs(X - Y) < fuzz): 45 | return 46 | self.fail("NumPy arrays {0} and {1} are not fuzzy equal (+- {2})".format(X, Y, fuzz)) 47 | 48 | def assertArraysEqualNotSame(self, a, b): 49 | """Assert that numpy arrays a and b are equal, but are not the same instances""" 50 | self.assertNotEqual(id(a), id(b)) 51 | self.assertApproxEqual(a, b) 52 | 53 | def assertArraysSame(self, a, b): 54 | # id(a) == id(b) doen't work for Cython memoryview arrays 55 | self.assertEqual(a.ndim, b.ndim) 56 | self.assertEqual(a.shape, b.shape) 57 | if a.ndim == 1: 58 | if a.shape[0] == 0: 59 | return # hard to test in this case 60 | orig_a, orig_b = a[0], b[0] 61 | a[0] = b[0] = 0 # reset 62 | a[0] = 1 # does it propagate to b? 63 | self.assertEqual(a[0], b[0]) 64 | a[0], b[0] = orig_a, orig_b # set back 65 | elif a.ndim == 2: 66 | if a.shape[0] == 0 or a.shape[1] == 0: 67 | return # hard to test 68 | orig_a, orig_b = a[0, 0], b[0, 0] 69 | a[0, 0] = b[0, 0] = 0 # reset 70 | a[0, 0] = 1 # does it propagate to b? 71 | self.assertEqual(a[0, 0], b[0, 0]) 72 | a[0, 0], b[0, 0] = orig_a, orig_b # set back 73 | else: 74 | self.fail("More than 2 or less than 1 dimensions not supported in this method") 75 | 76 | def assertRVsEqualNotSame(self, a, b): 77 | """Assert that :class:`~pybayes.pdfs.RV` objects a and b are equal, but 78 | are not the same instances neither shallow copies of themselves. 79 | 80 | RVs are special case during deepcopy - the RVComps should be referenced, 81 | not copied.""" 82 | self.assertNotEqual(id(a), id(b)) 83 | self.assertNotEqual(id(a.components), id(b.components)) 84 | self.assertEqual(a.name, b.name) # no need to test for id inequality - strings are immutable 85 | self.assertEqual(a.dimension, b.dimension) # ditto 86 | for (a_comp, b_comp) in zip(a.components, b.components): 87 | # equality for rv comps is defined as object instance identity 88 | self.assertEqual(a_comp, b_comp) 89 | -------------------------------------------------------------------------------- /pybayes/tests/test_filters.py: -------------------------------------------------------------------------------- 1 | """Tests for kalman""" 2 | 3 | from copy import copy, deepcopy 4 | 5 | import numpy as np 6 | 7 | import pybayes as pb 8 | from support import PbTestCase, stochastic 9 | 10 | 11 | class TestKalmanFilter(PbTestCase): 12 | """Tests for kalman filter""" 13 | 14 | def setUp(self): 15 | # synthetic parameters. May be completely mathematically invalid 16 | self.setup_1 = { # n = 2, k = 3, j = 4 17 | "A":np.array([[1., 2], [3, 4]]), # n*n 18 | "B":np.array([[1., 2, 3], [4, 5, 6]]), # n*k 19 | "C":np.array([[1., 2], [3, 4], [5, 6], [7, 8]]), # j*n 20 | "D":np.array([[1., 2, 3], [5, 6, 7], [9, 1, 2], [2, 3, 4]]), # j*k 21 | "Q":np.array([[2., 3], [4, 5]]), # n*n 22 | "R":np.array([[1., 2, 3, 4], [5, 6, 7, 8], [9, 1, 2, 3], [2, 3, 4, 5]]), # j*j 23 | "state_pdf":pb.GaussPdf(np.array([1., 2]), np.array([[1., 0], [0, 2]])) # n 24 | } 25 | self.setup_2 = { # n = 2, k = 1, j = 1 26 | "A":np.array([[1.0, -0.5],[1.0, 0.0]]), 27 | "B":np.array([[1.0],[0.1]]), 28 | "C":np.array([[1.0, 0.0]]), 29 | "D":np.array([[0.1]]), 30 | "Q":np.array([[0.2, 0.0],[0.0, 0.2]]), 31 | "R":np.array([[0.01]]), 32 | "state_pdf":pb.GaussPdf(np.array([0.0, 0.0]), np.array([[200.0, 0.0],[0.0, 200.0]])) 33 | } 34 | 35 | def test_init(self): 36 | k = pb.KalmanFilter(**self.setup_1) 37 | self.assertEqual(type(k), pb.KalmanFilter) 38 | l = pb.KalmanFilter(**self.setup_2) 39 | self.assertEqual(type(l), pb.KalmanFilter) 40 | 41 | def test_invalid_init(self): 42 | args = ["A", "B", "C", "D", "Q", "R", "state_pdf"] 43 | 44 | # invalid type: 45 | for arg in args: 46 | setup = self.setup_1.copy() 47 | setup[arg] = 125.65 48 | self.assertRaises(Exception, pb.KalmanFilter, **setup) 49 | 50 | # invalid dimension 51 | del args[6] # remove state_pdf 52 | for arg in args: 53 | setup = self.setup_1.copy() 54 | setup[arg] = np.array([[1.], [2.]]) 55 | self.assertRaises(ValueError, pb.KalmanFilter, **setup) 56 | gauss = pb.GaussPdf(np.array([1.]), np.array([[1.]])) 57 | setup = self.setup_1.copy() 58 | setup['state_pdf'] = gauss 59 | self.assertRaises(ValueError, pb.KalmanFilter, **setup) 60 | 61 | def test_bayes_evidence(self): 62 | k = pb.KalmanFilter(**self.setup_2) 63 | y = np.array([[4.1], [-0.2], [1.4], [-2.1]]) 64 | u = np.array([[4.8], [-0.3], [1.1], [-1.8]]) 65 | exp_mu = np.array([ 66 | [ 3.62004716, -0.46320771], 67 | [-0.16638519, 3.58787721], 68 | [ 1.21108425, 0.0224309 ], 69 | [-1.87141692, 0.98517451] 70 | ]) 71 | exp_var = np.array([ 72 | [ 0.00999960, 40.3342872], 73 | [ 0.00999029, 0.20999610], 74 | [ 0.00963301, 0.20962422], 75 | [ 0.00963191, 0.20930431] 76 | ]) 77 | # in: y_t -1, y_t, y_t + 1 78 | exp_evidences_log = np.array([ 79 | [ -3.68958564, -3.68287128, -3.68015356], 80 | [ -3.16749303, -2.75744797, -2.44453198], 81 | [ -2.69696927, -8.75357053, -18.48011903], 82 | [-10.17228316, -3.47352413, -0.45566728] 83 | ]) 84 | for i in range(4): 85 | k.bayes(y[i], u[i]) 86 | post = k.posterior() 87 | self.assertApproxEqual(post.mean(), exp_mu[i]) 88 | self.assertApproxEqual(post.variance(), exp_var[i]) 89 | evidences = np.array([k.evidence_log(y[i] - 1.), k.evidence_log(y[i]), k.evidence_log(y[i] + 1.)]) 90 | self.assertApproxEqual(evidences, exp_evidences_log[i]) 91 | 92 | def test_copy(self): 93 | """Test that copying KF works as expected""" 94 | o = pb.KalmanFilter(**self.setup_1) # original 95 | c = copy(o) # copy 96 | self.assertEqual(type(c), type(o)) 97 | 98 | self.assertNotEqual(id(o), id(c)) 99 | self.assertArraysSame(o.A, c.A) 100 | self.assertArraysSame(o.B, c.B) 101 | self.assertArraysSame(o.C, c.C) 102 | self.assertArraysSame(o.D, c.D) 103 | self.assertArraysSame(o.Q, c.Q) 104 | self.assertArraysSame(o.R, c.R) 105 | self.assertEqual(o.n, c.n) 106 | self.assertEqual(o.k, c.k) 107 | self.assertEqual(o.j, c.j) 108 | self.assertEqual(id(o.P), id(c.P)) 109 | self.assertEqual(id(o.S), id(c.S)) 110 | 111 | def test_deepcopy(self): 112 | """Test that deep copying KF works as expected""" 113 | o = pb.KalmanFilter(**self.setup_2) # original 114 | c = deepcopy(o) # copy 115 | self.assertEqual(type(c), type(o)) 116 | 117 | self.assertTrue(id(o) != id(c)) 118 | for (a, b) in [(o.A, c.A), (o.B, c.B), (o.C, c.C), (o.D, c.D), (o.Q, c.Q), (o.R, c.R)]: 119 | self.assertArraysEqualNotSame(a, b) 120 | # n, k, j do not need to be different as they are immutable 121 | self.assertEqual(o.n, c.n) 122 | self.assertEqual(o.k, c.k) 123 | self.assertEqual(o.j, c.j) 124 | self.assertTrue(id(o.P) != id(c.P)) 125 | self.assertArraysEqualNotSame(o.P.mu, c.P.mu) # this is better tested in 126 | self.assertArraysEqualNotSame(o.P.R, c.P.R) # GaussPdf deepcopy test, but wont hurt here 127 | self.assertTrue(id(o.S) != id(c.S)) 128 | 129 | 130 | class testParticleFilter(PbTestCase): 131 | """Tests for particle filter""" 132 | 133 | def setUp(self): 134 | init_pdf = pb.UniPdf(np.array([-5.]), np.array([5.])) 135 | p_xt_xtp = pb.MLinGaussCPdf(np.array([[2.]]), np.array([[1.]]), np.array([0.])) 136 | p_yt_xt = pb.MLinGaussCPdf(np.array([[1.]]), np.array([[1.]]), np.array([0.])) 137 | 138 | self.pf = pb.ParticleFilter(20, init_pdf, p_xt_xtp, p_yt_xt) 139 | 140 | def test_init(self): 141 | self.assertEqual(type(self.pf), pb.ParticleFilter) 142 | 143 | def test_bayes(self): 144 | # TODO: this test currently does little to verify that PF gives appropriate results 145 | #np.set_printoptions(linewidth=120, precision=2, suppress=True) 146 | for i in range(20): 147 | self.pf.bayes(np.array([i], dtype=float)) 148 | pdf = self.pf.posterior() 149 | #print "observation, mean:", i, pdf.mean()[0] 150 | -------------------------------------------------------------------------------- /pybayes/tests/test_wrappers_linalg.pxd: -------------------------------------------------------------------------------- 1 | # wrappers.linalg test needs this file, because in cython wrappers.linalg, there are only 2 | # cdefs, not cpdefs 3 | 4 | cimport cython 5 | 6 | cimport pybayes.wrappers._linalg as linalg 7 | 8 | 9 | ctypedef double[:, :] double_2D 10 | 11 | @cython.locals(A = double_2D, iA = double_2D) 12 | cpdef test_inv_func(self) 13 | 14 | @cython.locals(arr = double_2D) 15 | cpdef test_slogdet_func(self) 16 | 17 | @cython.locals(arr = double_2D, res = double_2D) 18 | cpdef test_cholesky_func(self) 19 | -------------------------------------------------------------------------------- /pybayes/tests/test_wrappers_linalg.py: -------------------------------------------------------------------------------- 1 | """Tests for wrappers._linalg""" 2 | 3 | import math 4 | 5 | import numpy as np 6 | 7 | import pybayes.wrappers._linalg as linalg 8 | from support import PbTestCase 9 | 10 | 11 | def test_inv_func(self): 12 | """Work-around so that this function can be annotated in .pxd""" 13 | # source data 14 | arrays = [ 15 | np.array([[ 2.]]), 16 | np.array([[ 0., 2.], [ 3., 0.]]), 17 | np.array([[ 1., -2.], [-4., 9.]]), 18 | np.array([[10., 11.], [100., 111.]]), # near singular 19 | np.array([[1., 2., -3.], [1., -2., 3.], [-1., 2., 3.]]) 20 | ] 21 | 22 | # test that A * inv(A) = I within machine precision 23 | for A in arrays: 24 | iA = linalg.inv(A) 25 | E = np.eye(A.shape[0]) 26 | E1 = np.dot(A, iA) 27 | E2 = np.dot(iA, A) 28 | self.assertApproxEqual(E1, E) 29 | self.assertApproxEqual(E2, E) 30 | 31 | def test_slogdet_func(self): 32 | """Work-around so that this function can be annotated in .pxd""" 33 | arr = np.array([[1., 2.], [-3., 4.]]) 34 | res = math.log(linalg.det(arr)) 35 | self.assertApproxEqual(res, 2.30258509299) 36 | 37 | def test_cholesky_func(self): 38 | """Work-around so that this function can be annotated in .pxd""" 39 | arr = np.array([[ 4., 12., -16.], 40 | [ 12., 37., -43.], 41 | [-16., -43., 98.]]) 42 | res = linalg.cholesky(arr) 43 | self.assertApproxEqual(np.dot(res, res.T), arr) 44 | 45 | 46 | class TestWrappersLinalg(PbTestCase): 47 | 48 | def test_inv(self): 49 | test_inv_func(self) 50 | 51 | def test_slogdet(self): 52 | """Test that we emulate slogdet correctly""" 53 | test_slogdet_func(self) 54 | 55 | def test_cholesky(self): 56 | test_cholesky_func(self) 57 | -------------------------------------------------------------------------------- /pybayes/tests/test_wrappers_numpy.pxd: -------------------------------------------------------------------------------- 1 | # wrappers.numpy test needs this file, because in cython wrappers.numpy, there are only 2 | # cdefs, not cpdefs 3 | 4 | cimport pybayes.wrappers._numpy as nw 5 | -------------------------------------------------------------------------------- /pybayes/tests/test_wrappers_numpy.py: -------------------------------------------------------------------------------- 1 | """Tests for wrappers._numpy""" 2 | 3 | import numpy as np 4 | 5 | import pybayes.wrappers._numpy as nw 6 | from support import PbTestCase 7 | 8 | 9 | class TestWrappersNumpy(PbTestCase): 10 | 11 | def test_vector(self): 12 | v = nw.vector(13) 13 | self.assertEqual(v.shape[0], 13) 14 | self.assertTrue(type(v[0]) in (float, np.float64)) 15 | 16 | def test_index_vector(self): 17 | v = nw.index_vector(13) 18 | self.assertEqual(v.shape[0], 13) 19 | self.assertTrue(type(v[0]) in (int, np.int64)) 20 | 21 | def test_matrix(self): 22 | m = nw.matrix(9, 11) 23 | self.assertEqual((m.shape[0], m.shape[1]), (9, 11)) 24 | self.assertTrue(type(m[0, 0]) in (float, np.float64)) 25 | -------------------------------------------------------------------------------- /pybayes/wrappers/__init__.py: -------------------------------------------------------------------------------- 1 | """Wrappers to ease dual (interpreted & compiled) mode development""" 2 | -------------------------------------------------------------------------------- /pybayes/wrappers/_linalg.pxd: -------------------------------------------------------------------------------- 1 | """Definitions for wrapper around numpy.linalg - cython version""" 2 | 3 | from ceygen.lu cimport * 4 | from ceygen.llt cimport * 5 | -------------------------------------------------------------------------------- /pybayes/wrappers/_linalg.py: -------------------------------------------------------------------------------- 1 | """Wrapper around numpy.linalg - python version""" 2 | 3 | from numpy.linalg import * 4 | -------------------------------------------------------------------------------- /pybayes/wrappers/_linalg.pyx: -------------------------------------------------------------------------------- 1 | """Wrapper around numpy.linalg - cython version""" 2 | -------------------------------------------------------------------------------- /pybayes/wrappers/_numpy.pxd: -------------------------------------------------------------------------------- 1 | """Definitions for wrapper around numpy - cython version""" 2 | 3 | from ceygen.core cimport * 4 | from ceygen.elemwise cimport * 5 | from ceygen.reductions cimport * 6 | from numpy cimport ndarray 7 | 8 | from pybayes.filters cimport KalmanFilter 9 | 10 | 11 | cdef double[:] vector(int size) 12 | cdef int[:] index_vector(int size) 13 | cdef int[:] index_range(int start, int stop) 14 | cdef double[:, :] matrix(int rows, int cols) 15 | 16 | cdef double[:] zeros(int size) 17 | 18 | ctypedef fused reindexable: 19 | double 20 | KalmanFilter 21 | 22 | cdef bint reindex_vv(reindexable[:] data, int[:] indices) except False 23 | cdef bint reindex_mv(reindexable[:, :] data, int[:] indices) except False 24 | 25 | cdef double[:] take_vv(double[:] data, int[:] indices, double[:] out = *) 26 | cdef bint put_vv(double[:] out, int[:] indices, double[:] data) except False 27 | -------------------------------------------------------------------------------- /pybayes/wrappers/_numpy.py: -------------------------------------------------------------------------------- 1 | """Wrapper around numpy - python version""" 2 | 3 | from numpy import * 4 | 5 | 6 | def vector(size): 7 | return empty(size) 8 | 9 | def index_vector(size): 10 | return empty(size, dtype=int) 11 | 12 | def index_range(start, stop): 13 | return arange(start, stop, dtype=int) 14 | 15 | def matrix(rows, cols): 16 | return empty((rows, cols)) 17 | 18 | def reindex_mv(data, indices): 19 | data[:] = data[indices] 20 | reindex_vv = reindex_mv 21 | 22 | # NumPy doesn't differentiate between vectors and matrices, Ceygen does: 23 | add_vv = add 24 | add_mm = add 25 | dot_vv = dot 26 | dot_vm = dot 27 | dot_mv = dot 28 | dot_mm = dot 29 | multiply_vs = multiply 30 | power_vs = power 31 | put_vv = put 32 | subtract_vv = subtract 33 | subtract_mm = subtract 34 | sum_v = sum 35 | take_vv = take 36 | -------------------------------------------------------------------------------- /pybayes/wrappers/_numpy.pyx: -------------------------------------------------------------------------------- 1 | """Wrapper around numpy - cython version""" 2 | 3 | # import numpy types and functions, override some as needed. 4 | # this file is special - it is used only in cython build, this can contain code 5 | # not callable from python etc. 6 | 7 | cimport ceygen.dtype as d 8 | # cython workaround: cannot import * 9 | from numpy import any, array, asarray, concatenate, cumsum, diag, exp, eye, ones, prod 10 | 11 | 12 | cdef double[:] vector(int size): 13 | return d.vector(size, 0) 14 | 15 | cdef int[:] index_vector(int size): 16 | return d.vector(size, 0) 17 | 18 | cdef int[:] index_range(int start, int stop): 19 | cdef int[:] v = d.vector(stop - start, 0) 20 | for i in range(stop - start): 21 | v[i] = start + i 22 | return v 23 | 24 | cdef double[:, :] matrix(int rows, int cols): 25 | return d.matrix(rows, cols, 0) 26 | 27 | cdef double[:] zeros(int size): 28 | ret = vector(size) 29 | ret[:] = 0 30 | return ret 31 | 32 | cdef bint reindex_vv(reindexable[:] data, int[:] indices) except False: 33 | assert data.shape[0] == indices.shape[0] 34 | cdef int newi 35 | datacopy = data.copy() 36 | for i in range(data.shape[0]): 37 | newi = indices[i] 38 | assert newi >= 0 and newi < data.shape[0] 39 | data[i] = datacopy[newi] 40 | return True 41 | 42 | 43 | cdef bint reindex_mv(reindexable[:, :] data, int[:] indices) except False: 44 | assert data.shape[0] == indices.shape[0] 45 | cdef int newi 46 | datacopy = data.copy() 47 | for i in range(data.shape[0]): 48 | newi = indices[i] 49 | assert newi >= 0 and newi < data.shape[0] 50 | data[i, :] = datacopy[newi, :] 51 | return True 52 | 53 | cdef double[:] take_vv(double[:] data, int[:] indices, double[:] out = None): 54 | if out is None: 55 | out = d.vector(indices.shape[0], 0) 56 | else: 57 | assert out.shape[0] >= indices.shape[0] 58 | for i in range(indices.shape[0]): 59 | out[i] = data[indices[i]] 60 | return out 61 | 62 | cdef bint put_vv(double[:] out, int[:] indices, double[:] data) except False: 63 | assert data.shape[0] >= indices.shape[0] 64 | for i in range(indices.shape[0]): 65 | out[indices[i]] = data[i] 66 | return True 67 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | from distutils.core import setup 4 | import os.path 5 | 6 | from support import determine_pybayes_version 7 | from support.dist import PyBayesDistribution 8 | 9 | 10 | dir = os.path.dirname(os.path.realpath(__file__)) 11 | version = determine_pybayes_version(dir=dir, fallback='0.3-post-nongit') 12 | with open(os.path.join(dir ,'README.rst')) as file: 13 | long_description = file.read() 14 | 15 | setup( 16 | packages=['pybayes', 'pybayes.stresses', 'pybayes.tests', 'pybayes.wrappers'], 17 | package_data={'pybayes.stresses':['data/stress_kalman_data.mat']}, 18 | distclass=PyBayesDistribution, 19 | 20 | # meta-data; see http://docs.python.org/distutils/setupscript.html#additional-meta-data 21 | name='PyBayes', 22 | version=version, 23 | author='Matěj Laitl', 24 | author_email='matej@laitl.cz', 25 | maintainer='Matěj Laitl', 26 | maintainer_email='matej@laitl.cz', 27 | url='https://github.com/strohel/PyBayes', 28 | description='Python library for recursive Bayesian estimation (Bayesian filtering)', 29 | long_description=long_description, 30 | # Note to myself: must manually upload on each release! 31 | download_url='https://github.com/downloads/strohel/PyBayes/PyBayes-'+version+'.tar.gz', 32 | platforms='cross-platform', 33 | license='MIT', 34 | classifiers=[ 35 | 'Development Status :: 3 - Alpha', 36 | 'Intended Audience :: Developers', 37 | 'Intended Audience :: Education', 38 | 'Intended Audience :: Science/Research', 39 | 'License :: OSI Approved :: MIT License', 40 | 'Operating System :: OS Independent', 41 | 'Programming Language :: Cython', 42 | 'Programming Language :: Python', 43 | 'Topic :: Scientific/Engineering :: Mathematics', 44 | 'Topic :: Scientific/Engineering :: Physics', 45 | 'Topic :: Software Development :: Libraries :: Python Modules', 46 | ], 47 | ) 48 | -------------------------------------------------------------------------------- /support/__init__.py: -------------------------------------------------------------------------------- 1 | """ 2 | A package with supportive Python modules that are not part of the PyBayes itself, but faciliate 3 | PyBayes building, installation, testing, stress-testing etc. 4 | """ 5 | 6 | import os 7 | import subprocess 8 | 9 | 10 | def determine_pybayes_version(dir, fallback): 11 | try: # try to get current version from git 12 | orig_dir = os.getcwd() 13 | os.chdir(dir) 14 | version = str(subprocess.check_output(['git', 'describe', '--dirty'])).lstrip('v').rstrip() 15 | os.chdir(orig_dir) 16 | except Exception as e: # CalledProcessError subclassed Exception directly 17 | print("Failed to determine version using git:", e) 18 | version = fallback 19 | return version 20 | -------------------------------------------------------------------------------- /support/debian/debian.changelog: -------------------------------------------------------------------------------- 1 | python-pybayes (0.3-1) stable; urgency=low 2 | 3 | * Initial Release 4 | 5 | -- Matej Laitl Mon, 3 Oct 2011 21:00:00 +0100 6 | -------------------------------------------------------------------------------- /support/debian/debian.control: -------------------------------------------------------------------------------- 1 | Source: python-pybayes 2 | Section: python 3 | Priority: optional 4 | Maintainer: Matej Laitl 5 | Build-Depends: debhelper (>= 4.1.16), libatlas-base-dev, python-numpy, python-dev (>= 2.6), cython (>=0.14.1) 6 | Package: python-pybayes 7 | Architecture: any 8 | Depends: ${python:Depends}, ${shlibs:Depends}, ${misc:Depends} 9 | Provides: ${python:Provides} 10 | Description: Python library for recursive Bayesian estimation (Bayesian filtering) 11 | PyBayes is an object-oriented Python library for recursive Bayesian 12 | estimation (Bayesian filtering) that is convenient to use. Already implemented are 13 | Kalman filter, particle filter and marginalized particle filter, all built atop of 14 | a light framework of probability density functions. PyBayes can optionally use Cython 15 | for lage speed gains (Cython build can be several times faster in some situations). 16 | -------------------------------------------------------------------------------- /support/debian/debian.rules: -------------------------------------------------------------------------------- 1 | #!/usr/bin/make -f 2 | # Sample debian/rules that uses debhelper. 3 | # GNU copyright 1997 to 1999 by Joey Hess. 4 | 5 | # Uncomment this to turn on verbose mode. 6 | export DH_VERBOSE=1 7 | 8 | # This is the debhelper compatibility version to use. 9 | export DH_COMPAT=5 10 | 11 | # So that build environment does not change (breaks build) 12 | export PYTHONDONTWRITEBYTECODE=x 13 | 14 | CFLAGS = -g 15 | ifneq (,$(findstring noopt,$(DEB_BUILD_OPTIONS))) 16 | CFLAGS += -O0 17 | else 18 | CFLAGS += -O2 19 | endif 20 | 21 | build: build-stamp 22 | build-stamp: 23 | dh_testdir 24 | 25 | # Add here commands to compile the package. 26 | python setup.py --use-cython=yes build test 27 | # --- end custom part for compiling 28 | 29 | touch build-stamp 30 | 31 | clean: 32 | dh_testdir 33 | dh_testroot 34 | rm -f build-stamp 35 | 36 | # Add here commands to clean up after the build process. 37 | python setup.py clean 38 | # --- end custom part for cleaning up 39 | 40 | dh_clean 41 | 42 | install: build 43 | dh_testdir 44 | dh_testroot 45 | dh_clean -k 46 | dh_installdirs 47 | 48 | # Add here commands to install the package 49 | # The DESTDIR Has To Be Exactly /usr/src/packages/BUILD/debian/ 50 | # --prefix=??? 51 | python setup.py install --root=/usr/src/packages/BUILD/debian/python-pybayes 52 | # --- end custom part for installing 53 | 54 | # Build architecture-independent files here. 55 | binary-indep: build install 56 | # We have nothing to do by default. 57 | 58 | # Build architecture-dependent files here. 59 | binary-arch: build install 60 | dh_testdir 61 | dh_testroot 62 | # dh_installdebconf 63 | dh_installdocs 64 | dh_installexamples 65 | dh_installmenu 66 | # dh_installlogrotate 67 | # dh_installemacsen 68 | # dh_installpam 69 | # dh_installmime 70 | # dh_installinit 71 | dh_installcron 72 | dh_installman 73 | dh_installinfo 74 | # dh_undocumented 75 | dh_installchangelogs 76 | dh_link 77 | dh_strip 78 | dh_compress 79 | dh_fixperms 80 | # dh_makeshlibs 81 | dh_installdeb 82 | # dh_perl 83 | dh_shlibdeps 84 | dh_gencontrol 85 | dh_md5sums 86 | dh_builddeb 87 | 88 | binary: binary-indep binary-arch 89 | .PHONY: build clean binary-indep binary-arch binary installing 90 | -------------------------------------------------------------------------------- /support/debian/python-pybayes.dsc: -------------------------------------------------------------------------------- 1 | Format: 1.0 2 | Source: python-pybayes 3 | Version: 0.3.9999-1 4 | Binary: python-pybayes 5 | Maintainer: Matej Laitl 6 | Architecture: any 7 | Build-Depends: debhelper (>= 4.1.16), libatlas-base-dev, python-numpy, python-dev (>= 2.6), cython (>=0.14.1) 8 | Files: 9 | ffffffffffffffffffffffffffffffff 0 PyBayes-0.3.tar.gz 10 | -------------------------------------------------------------------------------- /support/dist.py: -------------------------------------------------------------------------------- 1 | """ 2 | An extension to distutils' Distribution to handle Python/Cython build of PyBayes 3 | """ 4 | 5 | try: 6 | # 3.x 7 | from distutils.command.build_py import build_py_2to3 as build_py 8 | except ImportError: 9 | # 2.x 10 | from distutils.command.build_py import build_py 11 | from distutils.dist import Distribution 12 | from distutils.errors import DistutilsOptionError 13 | import distutils.log as log 14 | from distutils.util import strtobool 15 | import os 16 | 17 | from .dist_cmd_build import build 18 | from .dist_cmd_build_prepare import build_prepare 19 | from .dist_cmd_stress import stress 20 | from .dist_cmd_test import test 21 | 22 | 23 | class PyBayesDistribution(Distribution): 24 | """An extension to distutils' Distribution that provides Cython/Python build switching etc.""" 25 | 26 | def __init__(self, attrs=None): 27 | Distribution.__init__(self, attrs) 28 | self.use_cython = None 29 | self.profile = False 30 | if not self.ext_modules: 31 | self.ext_modules = [] 32 | 33 | # it is better to define command classes here, so they are available in --help text 34 | self.cmdclass['test'] = test 35 | self.cmdclass['stress'] = stress 36 | self.cmdclass['build_prepare'] = build_prepare 37 | self.cmdclass['build_py'] = build_py 38 | 39 | self.global_options += [ 40 | ('use-cython=', None, "use Cython to make faster binary python modules (choices: " 41 | + "yes/no; default: autodetect)"), 42 | ('profile=', None, 'embed profiling information into Cython build (choices: ' 43 | + 'yes/no; default: no)'), 44 | ] 45 | 46 | def has_ext_modules(self): 47 | if self.use_cython: 48 | return True 49 | return Distribution.has_ext_modules(self) 50 | 51 | def parse_command_line(self): 52 | """We need to process the options once command line is parsed""" 53 | ret = Distribution.parse_command_line(self) 54 | if ret: 55 | self.finalize_command_line() 56 | return ret 57 | 58 | def finalize_command_line(self): 59 | if self.profile not in (True, False): 60 | self.profile = bool(strtobool(self.profile)) 61 | if self.use_cython is None: 62 | self.use_cython = self._find_cython() 63 | if self.use_cython: 64 | log.info("Cython and NumPy found, enabling optimised Cython build.") 65 | else: 66 | log.info("Cython or NumPy was not found on your system. Falling back to pure") 67 | log.info("python mode which may be somehow slower.") 68 | elif self.use_cython not in (True, False): 69 | requested = strtobool(self.use_cython) 70 | if requested and not self._find_cython(): 71 | raise DistutilsOptionError("Cython build was requested but no or too old Cython" 72 | + " and/or NumPy was found on your system.") 73 | self.use_cython = bool(requested) 74 | if self.use_cython: 75 | log.debug("Cython build requested and Cython and NumPy found.") 76 | else: 77 | log.debug("Pure Python build requested, not searching for Cython.") 78 | if self.use_cython: 79 | self.finalize_cython_options() 80 | 81 | def finalize_cython_options(self): 82 | self.cmdclass['build'] = build 83 | self.cmdclass['build_ext'] = self.build_ext 84 | 85 | # .pyc files just litter site-packages in Cython build 86 | install = self.get_command_obj('install') 87 | install.compile = 0 88 | install.optimise = 0 89 | 90 | def _find_cython(self): 91 | """Returns true if sufficient version of Cython in found on system, false otherwise. 92 | 93 | If true is returned, also sets some variables useful for Cython build 94 | """ 95 | # autodetect (or check for) cython 96 | try: 97 | from Cython.Distutils import build_ext 98 | from Cython.Distutils.extension import Extension 99 | except ImportError as e: 100 | log.debug("Cython was not found: {0}".format(e)) 101 | return False 102 | self.build_ext = build_ext 103 | self.Extension = Extension 104 | 105 | # determine path to NumPy C header files 106 | try: 107 | import numpy 108 | except ImportError as e: 109 | log.warn("Error importing numpy: {0}".format(e)) 110 | log.warn("Cython was found on your system, but NumPy was not. NumPy is needed") 111 | log.warn("build-time for cython builds and runtime for all builds. Falling back") 112 | log.warn("to pure Python build.") 113 | return False 114 | self.numpy_include_dir = numpy.get_include() 115 | return True 116 | -------------------------------------------------------------------------------- /support/dist_cmd_build.py: -------------------------------------------------------------------------------- 1 | """ 2 | An extension distutils' build to handle Python/Cython build of PyBayes 3 | """ 4 | 5 | from distutils.command.build import build as orig_build 6 | 7 | 8 | class build(orig_build): 9 | """Introduce additional build step to inject Cython extensions""" 10 | 11 | def finalize_options(self): 12 | orig_build.finalize_options(self) 13 | # prepend our custom command 14 | self.sub_commands = [('build_prepare', None)] + self.sub_commands 15 | self.build_lib = self.build_platlib 16 | if self.distribution.profile: 17 | # so that profiling and non-profiling builds do not clash 18 | self.build_lib += '-profile' 19 | # Cython would otherwise think that .c files need not be regenerated: 20 | self.build_temp += '-profile' 21 | -------------------------------------------------------------------------------- /support/dist_cmd_build_prepare.py: -------------------------------------------------------------------------------- 1 | """ 2 | An additional subcommand to distutils' build to handle Python/Cython build of PyBayes 3 | """ 4 | 5 | from distutils.cmd import Command 6 | from distutils.errors import DistutilsSetupError 7 | import distutils.log as log 8 | from distutils.sysconfig import get_config_var 9 | from glob import glob 10 | import os 11 | import string 12 | 13 | 14 | class build_prepare(Command): 15 | """Additional build step that is used to add Cython Extension per each module that is specified""" 16 | 17 | description = 'scan defined packages for python modules and inject Cython module per each' 18 | 19 | def initialize_options(self): 20 | self.ext_options = {} # options common to all extensions 21 | self.ext_options['include_dirs'] = [self.distribution.numpy_include_dir] 22 | self.ext_options['extra_compile_args'] = ['-O2'] 23 | #self.ext_options['extra_link_args'] = ['-Wl,-O1'] 24 | self.ext_options['pyrex_c_in_temp'] = True # do not pollute source directory with .c files 25 | self.ext_options['pyrex_directives'] = { 26 | 'profile':self.distribution.profile, 27 | 'infer_types':True, 28 | "binding": False, # default was changed to True in Cython commit 621dbe6403 and it 29 | # breaks the build. I don't know what it means, it's undocumented. 30 | } 31 | self.deps = [] # .pxd dependencies for injected packages 32 | 33 | def finalize_options(self): 34 | # these options are passed through global distribution 35 | dist = self.distribution 36 | 37 | # these are just aliases to distribution variables 38 | self.packages = self.distribution.packages 39 | self.py_modules = self.distribution.py_modules 40 | self.package_data = self.distribution.package_data 41 | self.package_dir = {} 42 | if self.distribution.package_dir: 43 | for name, path in self.distribution.package_dir.items(): 44 | self.package_dir[name] = convert_path(path) 45 | 46 | if self.py_modules: 47 | raise DistutilsSetupError("PyBayes-tweaked distutils doesn't support nonempty `py_modules`") 48 | if not self.packages: 49 | raise DistutilsSetupError("PyBayes-tweaked distutils doesn't support nempty `packages`") 50 | 51 | def run(self): 52 | build_py = self.distribution.get_command_obj('build_py') 53 | self.get_package_dir = build_py.get_package_dir # borrow a method from build_py 54 | self.check_package = build_py.check_package # ditto 55 | 56 | for package in self.packages: 57 | package_dir = self.get_package_dir(package) 58 | self.inject_package_modules(package, package_dir) 59 | 60 | self.update_dependencies() 61 | 62 | def inject_package_modules (self, package, package_dir): 63 | """This is our purpose and a hack - we create Cython extensions here""" 64 | self.check_package(package, package_dir) 65 | py_files = glob(os.path.join(package_dir, "*.py")) 66 | pyx_files = glob(os.path.join(package_dir, "*.pyx")) 67 | for pyx_file in pyx_files: # subtract py files that also have a pyx file 68 | corres_py_file = pyx_file[:-3]+"py" # corresponding .py file 69 | if corres_py_file in py_files: 70 | del py_files[py_files.index(corres_py_file)] 71 | pxd_files = glob(os.path.join(package_dir, "*.pxd")) 72 | self.deps += pxd_files 73 | setup_script = os.path.abspath(self.distribution.script_name) 74 | 75 | if package not in self.package_data: 76 | self.package_data[package] = [] 77 | # per-package data have to have path relative to their package 78 | self.package_data[package] += [os.path.basename(pxd_file) for pxd_file in pxd_files] 79 | 80 | for f in py_files + pyx_files: 81 | basename = os.path.basename(f) 82 | if os.path.abspath(f) == setup_script: 83 | self.debug_print("excluding %s" % setup_script) 84 | continue 85 | if basename == '__init__.py': 86 | # otherwise import package (`import pybayes`) does not really work 87 | continue 88 | if basename == '__main__.py': 89 | # otherwise `python -m pybayes.tests` does not really work 90 | continue 91 | 92 | module = os.path.splitext(f)[0].replace("/", ".") 93 | self.inject_extension(module, f) 94 | 95 | def inject_extension(self, module, f): 96 | log.info("injecting Cython extension {0} (module {1})".format(f, module)) 97 | self.distribution.ext_modules.append( 98 | self.distribution.Extension(module, [f], **self.ext_options) 99 | ) 100 | 101 | def update_dependencies(self): 102 | """Update dependencies of all already injected extensions""" 103 | for module in self.distribution.ext_modules: 104 | module.sources += self.deps 105 | -------------------------------------------------------------------------------- /support/dist_cmd_stress.py: -------------------------------------------------------------------------------- 1 | """ 2 | A custom command for distutils to facilitate stress-testing of PyBayes 3 | """ 4 | 5 | from distutils.cmd import Command 6 | from distutils.errors import DistutilsExecError 7 | import distutils.log as log 8 | from os.path import abspath, dirname, join 9 | import sys 10 | import unittest 11 | 12 | 13 | class stress(Command): 14 | """Stress-test PyBayes in the build directory""" 15 | 16 | description = 'run stress-suite of PyBayes within build directory' 17 | user_options = [] 18 | 19 | def initialize_options(self): 20 | self.build_lib = None 21 | 22 | def finalize_options(self): 23 | self.set_undefined_options('build', ('build_lib', 'build_lib')) 24 | 25 | def run(self): 26 | self.run_command('build') # build if not alredy run 27 | orig_path = sys.path[:] 28 | try: 29 | tests_path = abspath(self.build_lib) 30 | sys.path.insert(0, tests_path) 31 | import pybayes.stresses 32 | if dirname(pybayes.stresses.__file__) != join(tests_path, 'pybayes', 'stresses'): 33 | raise Exception("Expected that imported pybayes.stresses would be from " 34 | + "{0}, but it was from {1} instead".format(tests_path, 35 | dirname(pybayes.tests.__file__))) 36 | suite = unittest.TestLoader().loadTestsFromModule(pybayes.stresses) 37 | result = unittest.TextTestRunner(verbosity=self.verbose).run(suite) 38 | if not result.wasSuccessful(): 39 | raise Exception("There were test failures") 40 | except Exception as e: 41 | raise DistutilsExecError(e) 42 | finally: 43 | sys.path = orig_path 44 | -------------------------------------------------------------------------------- /support/dist_cmd_test.py: -------------------------------------------------------------------------------- 1 | """ 2 | A custom command for distutils to facilitate testing of PyBayes 3 | """ 4 | 5 | from distutils.cmd import Command 6 | from distutils.errors import DistutilsExecError 7 | import distutils.log as log 8 | from os.path import abspath, dirname, join 9 | import sys 10 | import unittest 11 | 12 | 13 | class test(Command): 14 | """Test PyBayes in the build directory""" 15 | 16 | description = 'run unit test-suite of PyBayes within build directory' 17 | user_options = [] 18 | 19 | def initialize_options(self): 20 | self.build_lib = None 21 | 22 | def finalize_options(self): 23 | self.set_undefined_options('build', ('build_lib', 'build_lib')) 24 | 25 | def run(self): 26 | self.run_command('build') # build if not alredy run 27 | orig_path = sys.path[:] 28 | try: 29 | tests_path = abspath(self.build_lib) 30 | sys.path.insert(0, tests_path) 31 | import pybayes.tests 32 | if dirname(pybayes.tests.__file__) != join(tests_path, 'pybayes', 'tests'): 33 | raise Exception("Expected that imported pybayes.tests would be from " 34 | + "{0}, but it was from {1} instead".format(tests_path, 35 | dirname(pybayes.tests.__file__))) 36 | suite = unittest.TestLoader().loadTestsFromModule(pybayes.tests) 37 | result = unittest.TextTestRunner(verbosity=self.verbose).run(suite) 38 | if not result.wasSuccessful(): 39 | raise Exception("There were test failures") 40 | except Exception as e: 41 | raise DistutilsExecError(e) 42 | finally: 43 | sys.path = orig_path 44 | -------------------------------------------------------------------------------- /support/rpm/python-pybayes.spec: -------------------------------------------------------------------------------- 1 | # 2 | # spec file for package python-pybayes 3 | # 4 | # Copyright (c) 2011 SUSE LINUX Products GmbH, Nuernberg, Germany. 5 | # 6 | # All modifications and additions to the file contributed by third parties 7 | # remain the property of their copyright owners, unless otherwise agreed 8 | # upon. The license for this file, and modifications and additions to the 9 | # file, is the same license as for the pristine package itself (unless the 10 | # license for the pristine package is not an Open Source License, in which 11 | # case the license is the MIT License). An "Open Source License" is a 12 | # license that conforms to the Open Source Definition (Version 1.9) 13 | # published by the Open Source Initiative. 14 | 15 | # Please submit bugfixes or comments via http://bugs.opensuse.org/ 16 | # 17 | 18 | 19 | Name: python-pybayes 20 | Version: 0.3.9999 21 | Release: 0 22 | Url: https://github.com/strohel/PyBayes 23 | Summary: Python library for recursive Bayesian estimation (Bayesian filtering) 24 | License: MIT 25 | Group: Development/Languages/Python 26 | Source: https://github.com/downloads/strohel/PyBayes/PyBayes-%{version}.tar.gz 27 | BuildRoot: %{_tmppath}/%{name}-%{version}-build 28 | 29 | BuildRequires: python-devel >= 2.6 30 | BuildRequires: python-cython >= 0.14.1 31 | 32 | %if 0%{?fedora} 33 | BuildRequires: numpy >= 1.5.0 34 | %else 35 | BuildRequires: python-numpy-devel >= 1.5.0 36 | %endif 37 | 38 | # following provides both clapack.h, cblas.h, cblas.so and lapack.so 39 | BuildRequires: atlas-devel 40 | 41 | %if 0%{?suse_version} 42 | %py_requires 43 | %endif 44 | 45 | %{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")} 46 | 47 | %description 48 | PyBayes is an object-oriented Python library for recursive Bayesian estimation (Bayesian filtering) 49 | that is convenient to use. Already implemented are Kalman filter, particle filter and marginalized 50 | particle filter, all built atop of a light framework of probability density functions. PyBayes can 51 | optionally use Cython for lage speed gains (Cython build can be several times faster in some 52 | situations). 53 | 54 | %prep 55 | %setup -q -n PyBayes-%{version} 56 | 57 | %build 58 | python setup.py --use-cython=yes build build_prepare --blas-lib cblas --lapack-lib lapack_atlas 59 | 60 | %check 61 | python setup.py --use-cython=yes test 62 | 63 | %install 64 | python setup.py install --prefix=%{_prefix} --root=%{buildroot} 65 | 66 | %files 67 | %defattr(-,root,root,-) 68 | %doc README.rst HACKING.rst ChangeLog.rst 69 | %{python_sitelib}/* 70 | 71 | %changelog 72 | -------------------------------------------------------------------------------- /thesis/.gitignore: -------------------------------------------------------------------------------- 1 | *.kilepr 2 | *.log 3 | *.backup 4 | *.aux 5 | *.toc 6 | *.out 7 | *.brf 8 | *.bbl 9 | *.blg 10 | *.nav 11 | *.snm 12 | *.dvi 13 | *.lof 14 | 15 | thesis.pdf 16 | -------------------------------------------------------------------------------- /thesis/KF.fig: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/KF.fig -------------------------------------------------------------------------------- /thesis/KF.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/KF.pdf -------------------------------------------------------------------------------- /thesis/KFPerf.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/KFPerf.pdf -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/.gitignore: -------------------------------------------------------------------------------- 1 | *.so 2 | build/ 3 | *.a 4 | 5 | #/integrate_cython.c # this file is generated, however distributed so that live cython is not needed 6 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/README: -------------------------------------------------------------------------------- 1 | How to compile cython module and C version? 2 | 3 | 1. cython must be installed, you need version >= 0.14.2 so that prange is supported 4 | 2. compile C version: 5 | $ gcc -O2 -fPIC -fopenmp -c integrate_c.c -o libintegrate_c.a 6 | 2. issue: 7 | $ ./setup.py build_ext -i 8 | 3. integrate_cython.so should appear next to integrate_cython.pyx 9 | 10 | How to run? 11 | 12 | 1. ./run.py (edit tests list to enable/disable tests) 13 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/edgy-results.ods: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/benchmark_c_cy_py/edgy-results.ods -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/integrate_c.c: -------------------------------------------------------------------------------- 1 | #include "integrate_c.h" /* just to assure that declarations match */ 2 | 3 | #include 4 | #include 5 | 6 | double f(double x) 7 | { 8 | return x*x; 9 | } 10 | 11 | double lib_integrate_c(double a, double b, int N) 12 | { 13 | double s = 0.0; 14 | double dx = (b-a)/N; 15 | int i; 16 | 17 | if(dx == 0.0) { 18 | fprintf(stderr, "dx == 0!\n"); 19 | return 0.0; 20 | } 21 | 22 | for(i = 0; i < N; i++) { 23 | s += f(a + (i + 1./2.)*dx)*dx; 24 | } 25 | return s; 26 | } 27 | 28 | double lib_integrate_c_omp(double a, double b, int N) 29 | { 30 | double s = 0.0; 31 | double dx = (b-a)/N; 32 | int i; 33 | 34 | 35 | if(dx == 0.0) { 36 | fprintf(stderr, "dx == 0!\n"); 37 | return 0.0; 38 | } 39 | 40 | #pragma omp parallel for reduction(+:s) lastprivate(a) 41 | for(i = 0; i < N; i++) { 42 | s += f(a + (i + 1./2.)*dx)*dx; 43 | } 44 | return s; 45 | } 46 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/integrate_c.h: -------------------------------------------------------------------------------- 1 | double lib_integrate_c(double a, double b, int N); 2 | double lib_integrate_c_omp(double a, double b, int N); 3 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/integrate_cython.pyx: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | #cython: boundscheck=False 3 | #cython: wraparound=False 4 | 5 | from cython.parallel cimport prange 6 | 7 | cdef extern from "integrate_c.h": 8 | double lib_integrate_c(double a, double b, int N) 9 | double lib_integrate_c_omp(double a, double b, int N) 10 | 11 | cpdef double f(double x) nogil: 12 | return x*x 13 | 14 | cpdef integrate(a, b, N): 15 | s = 0 16 | dx = (b-a)/N 17 | 18 | if dx == 0: 19 | print "dx == 0!" 20 | return 0 21 | 22 | for i in xrange(N): 23 | s += f(a + (i + 1./2.)*dx)*dx 24 | return s 25 | 26 | cpdef double integrate_typed(double a, double b, int N): 27 | cdef double s = 0 28 | cdef double dx = (b-a)/N 29 | cdef int i 30 | 31 | if dx == 0: 32 | print "dx == 0!" 33 | return 0 34 | 35 | for i in xrange(N): 36 | s += f(a + (i + 1./2.)*dx)*dx 37 | return s 38 | 39 | cpdef double integrate_omp(double a, double b, int N): 40 | cdef double s = 0 41 | cdef double dx = (b-a)/N 42 | cdef int i 43 | 44 | if dx == 0: 45 | print "dx == 0!" 46 | return 0 47 | 48 | for i in prange(N, nogil=True, schedule=guided): 49 | s += f(a + (i + 1./2.)*dx)*dx 50 | return s 51 | 52 | cpdef double integrate_c(double a, double b, int N): 53 | return lib_integrate_c(a, b, N) 54 | 55 | cpdef double integrate_c_omp(double a, double b, int N): 56 | return lib_integrate_c_omp(a, b, N) 57 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/integrate_python.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | def f(x): 4 | return x*x 5 | 6 | def integrate(a, b, N): 7 | s = 0 8 | dx = (b-a)/N 9 | 10 | if dx == 0: 11 | print "dx == 0!" 12 | return 0 13 | 14 | for i in xrange(N): 15 | s += f(a + (i + 1./2.)*dx)*dx 16 | return s 17 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/results-esprimo-optimised.txt: -------------------------------------------------------------------------------- 1 | Integration parameters (a, b, N): (0.0, 3.0, 300000000) 2 | 3 | cython_typed_openmp integrate() = 8.999999955 4 | Time spent: 0.536821126938 5 | 6 | cython_typed integrate() = 8.999999955 7 | Time spent: 0.876391887665 8 | 9 | cython integrate() = 8.999999955 10 | Time spent: 98.4380710125 11 | 12 | python integrate() = 8.999999955 13 | Time spent: 270.829223871 14 | 15 | Relative speedups (2-core Intel Core 2 Duo processor): 16 | 17 | cython_typed_openmp/python: 504.505523872 18 | cython_typed/python: 309.02753401 19 | cython/python: 2.75126504497 20 | 21 | cython_typed_openmp/cython: 183.372199924 22 | cython_typed/cython: 112.321978784 23 | 24 | cython_typed_openmp/cython_typed: 1.63255848864 25 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/results-esprimo-unoptimised.txt: -------------------------------------------------------------------------------- 1 | Integration parameters (a, b, N): (0.0, 3.0, 300000000) 2 | 3 | cython_typed_openmp integrate() = 8.999999955 4 | Time spent: 3.37605309486 5 | 6 | cython_typed integrate() = 8.999999955 7 | Time spent: 6.06836104393 8 | 9 | cython integrate() = 8.999999955 10 | Time spent: 113.26262188 11 | 12 | python integrate() = 8.999999955 13 | Time spent: 268.439502954 14 | 15 | Relative speedups (2-core Intel Core 2 Duo processor): 16 | 17 | cython_typed_openmp/python: 79.5128202702 18 | cython_typed/python: 44.2359149383 19 | cython/python: 2.37006258993 20 | 21 | cython_typed_openmp/cython: 33.5488271947 22 | cython_typed/cython: 18.6644500978 23 | 24 | cython_typed_openmp/cython_typed: 1.79747204011 25 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/results-kmlinux-optimised.txt: -------------------------------------------------------------------------------- 1 | Numerical integration from 0.000000 to 3.000000 of x^2 with 1000000000 steps: 2 | 3 | cython_typed_openmp: result = 9.000000; real time = 0.451694s; cpu time = 7.160000s 4 | cython_typed: result = 9.000000; real time = 6.573066s; cpu time = 7.110000s 5 | cython: result = 9.000000; real time = 202.516819s; cpu time = 202.490000s 6 | python: result = 9.000000; real time = 617.069662s; cpu time = 617.020000s 7 | 8 | Relative speedups (16-core AMD Opteron): 9 | 10 | cython_typed_openmp/python: 1366.12318545 11 | cython_typed/python: 93.8785130795 12 | cython/python: 3.04700451617 13 | 14 | cython_typed_openmp/cython: 448.349576837 15 | cython_typed/cython: 30.8100997492 16 | 17 | cython_typed_openmp/cython_typed: 14.5520326285 18 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/results-kmlinux-unoptimised.txt: -------------------------------------------------------------------------------- 1 | Integration parameters (a, b, N): (0.0, 3.0, 300000000) 2 | 3 | cython_typed_openmp integrate() = 8.999999955 4 | Time spent: 0.526089906693 5 | 6 | cython_typed integrate() = 8.999999955 7 | Time spent: 8.34564685822 8 | 9 | cython integrate() = 8.999999955 10 | Time spent: 73.0280399323 11 | 12 | python integrate() = 8.999999955 13 | Time spent: 195.182024956 14 | 15 | Relative speedups (16-core AMD Opteron): 16 | 17 | cython_typed_openmp/python: 371.005074366 18 | cython_typed/python: 23.3872854042 19 | cython/python: 2.67269976213 20 | 21 | cython_typed_openmp/cython: 138.812851194 22 | cython_typed/cython: 8.75043494805 23 | 24 | cython_typed_openmp/cython_typed: 15.8635373005 25 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/run.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | try: 5 | import integrate_cython as c 6 | except ImportError as e: 7 | print "Failed to import integrate_cython, cython tests wont be available:", e 8 | c = None 9 | import integrate_python as p 10 | import time 11 | 12 | # Edit parameters here: 13 | 14 | params = 0.0, 3.0, 200*10**6 # from, to, N (number of steps) 15 | 16 | # Comment/Uncomment individual tests here> 17 | 18 | tests = [] 19 | if c: 20 | tests += [("c_omp", c.integrate_c_omp), 21 | ("cy_typed_omp", c.integrate_omp), 22 | ("c", c.integrate_c), 23 | ("cython_typed", c.integrate_typed), 24 | ("cython", c.integrate)] 25 | tests += [("python", p.integrate)] 26 | 27 | 28 | # No need to edit edit below here 29 | 30 | times = {} 31 | 32 | print "Numerical integration from {0} to {1} of x^2 with {2} steps:".format(*params) 33 | print 34 | 35 | for (name, func) in tests: 36 | start_time = time.time() 37 | start_clock = time.clock() 38 | result = func(*params) 39 | times[name] = (time.time() - start_time, time.clock() - start_clock) 40 | print "{0:>12}: result = {1}; real time = {2}s; cpu time = {3}s".format(name, result, times[name][0], times[name][1]) 41 | 42 | print 43 | print "Relative speedups:" 44 | 45 | for i in range(len(tests) - 1, 0, -1): 46 | print 47 | iname = tests[i][0] 48 | for j in range(i-1, -1, -1): 49 | jname = tests[j][0] 50 | print jname + "/" + iname + ":", times[iname][0]/times[jname][0] 51 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | from distutils.core import setup 5 | from distutils.extension import Extension 6 | from Cython.Distutils import build_ext 7 | 8 | # extra_compile_args=["-O2", "-march=core2", "-fopenmp" 9 | 10 | setup( 11 | cmdclass = {'build_ext': build_ext}, 12 | ext_modules = [ 13 | Extension( 14 | "integrate_cython", 15 | ["integrate_cython.pyx"], 16 | libraries=["integrate_c"], 17 | extra_compile_args=["-O2", "-fopenmp"], 18 | extra_link_args=["-Wl,-O1", "-fopenmp", "-L./"] 19 | ) 20 | ] 21 | ) 22 | -------------------------------------------------------------------------------- /thesis/benchmark_c_cy_py/zavislost_na_gcc_prepinacich.txt: -------------------------------------------------------------------------------- 1 | -02 -march=core2: 2 | cython_typed_openmp: result = 8.9999999865; real time = 1.69873094559s; cpu time = 3.12s 3 | cython_typed: result = 8.9999999865; real time = 2.86416888237s; cpu time = 2.99s 4 | 5 | -02: 6 | cython_typed_openmp: result = 8.9999999865; real time = 1.66599702835s; cpu time = 3.07s 7 | cython_typed: result = 8.9999999865; real time = 2.87195682526s; cpu time = 2.99s 8 | 9 | -01: 10 | cython_typed_openmp: result = 8.9999999865; real time = 2.95676016808s; cpu time = 5.73s 11 | cython_typed: result = 8.9999999865; real time = 5.25587296486s; cpu time = 5.4s 12 | 13 | : 14 | cython_typed_openmp: result = 8.9999999865; real time = 11.1103010178s; cpu time = 21.34s 15 | cython_typed: result = 8.9999999865; real time = 19.8046140671s; cpu time = 19.94s 16 | -------------------------------------------------------------------------------- /thesis/benchmark_c_java/.gitignore: -------------------------------------------------------------------------------- 1 | Mandelbrot.class 2 | test 3 | -------------------------------------------------------------------------------- /thesis/benchmark_c_java/README: -------------------------------------------------------------------------------- 1 | How to run? 2 | 3 | 1. compile java code into bytecode: 4 | $ javac test.java 5 | 6 | 2. compile C code: 7 | $ gcc -O2 test.c -o test # you may replace -O2 with different optimisation flags 8 | 9 | 3. Run them: 10 | $ java -XX:CompileThreshold= Mendelbrot # set low threshold to force JIT compilation, 11 | # set very high threshold to disable JIT 12 | $ ./test 13 | 14 | You may want tu suppres standard error output to see only measured times: 15 | $ ./test 2>/dev/null 16 | -------------------------------------------------------------------------------- /thesis/benchmark_c_java/test.c: -------------------------------------------------------------------------------- 1 | // by Erik Wrenholt 2 | #include 3 | #include 4 | 5 | #define BAILOUT 16 6 | #define MAX_ITERATIONS 1000 7 | 8 | int mandelbrot(float x, float y) { 9 | float cr = y - 0.5; 10 | float ci = x; 11 | float zi = 0.0; 12 | float zr = 0.0; 13 | int i = 0; 14 | while(1) { 15 | i ++; 16 | float temp = zr * zi; 17 | float zr2 = zr * zr; 18 | float zi2 = zi * zi; 19 | zr = zr2 - zi2 + cr; 20 | zi = temp + temp + ci; 21 | if (zi2 + zr2 > BAILOUT) 22 | return i; 23 | if (i > MAX_ITERATIONS) 24 | return 0; 25 | } 26 | } 27 | 28 | void run2() { 29 | int x,y; 30 | for (y = -39; y < 39; y++) { 31 | fputs("\n", stderr); 32 | for (x = -39; x < 39; x++) { 33 | int i = mandelbrot(x/40.0, y/40.0); 34 | if (i==0) 35 | fputs("*", stderr); 36 | else 37 | fputs(" ", stderr); 38 | } 39 | } 40 | fputs("\n", stderr); 41 | } 42 | 43 | void run() { 44 | struct timeval aTv; 45 | gettimeofday(&aTv, NULL); 46 | long init_time = aTv.tv_sec; 47 | long init_usec = aTv.tv_usec; 48 | int i; 49 | for (i = 0; i < 100; i++) 50 | run2(); 51 | gettimeofday(&aTv,NULL); 52 | double query_time = (aTv.tv_sec - init_time) + (double)(aTv.tv_usec - init_usec)/1000000.0; 53 | printf ("C Elapsed %0.2f\n", query_time); 54 | } 55 | 56 | int main (int argc, const char * argv[]) { 57 | run(); 58 | run(); 59 | run(); 60 | } 61 | -------------------------------------------------------------------------------- /thesis/benchmark_c_java/test.java: -------------------------------------------------------------------------------- 1 | // by Erik Wrenholt 2 | import java.util.*; 3 | 4 | class Mandelbrot { 5 | static int BAILOUT = 16; 6 | static int MAX_ITERATIONS = 1000; 7 | 8 | private static int iterate(float x, float y) { 9 | float cr = y-0.5f; 10 | float ci = x; 11 | float zi = 0.0f; 12 | float zr = 0.0f; 13 | int i = 0; 14 | while (true) { 15 | i++; 16 | float temp = zr * zi; 17 | float zr2 = zr * zr; 18 | float zi2 = zi * zi; 19 | zr = zr2 - zi2 + cr; 20 | zi = temp + temp + ci; 21 | if (zi2 + zr2 > BAILOUT) 22 | return i; 23 | if (i > MAX_ITERATIONS) 24 | return 0; 25 | } 26 | } 27 | 28 | public static void run2() { 29 | int x,y; 30 | for (y = -39; y < 39; y++) { 31 | System.err.print("\n"); 32 | for (x = -39; x < 39; x++) { 33 | if (iterate(x/40.0f,y/40.0f) == 0) 34 | System.err.print("*"); 35 | else 36 | System.err.print(" "); 37 | } 38 | } 39 | } 40 | 41 | public static void run() { 42 | Date d1 = new Date(); for (int i = 0; i < 100; i++) run2(); 43 | Date d2 = new Date(); 44 | long diff = d2.getTime() - d1.getTime(); 45 | System.out.println("Java Elapsed " + diff/1000.0f); 46 | } 47 | 48 | public static void main(String args[]) { 49 | run(); 50 | run(); 51 | run(); 52 | } 53 | } 54 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/big_cython.txt: -------------------------------------------------------------------------------- 1 | stress_kalman(): 2 | Time spent: 0.511182069778s real time; 0.51s CPU time 3 | stress_kalman(): 4 | Time spent: 0.510687828064s real time; 0.51s CPU time 5 | stress_kalman(): 6 | Time spent: 0.518685102463s real time; 0.52s CPU time 7 | stress_kalman(): 8 | Time spent: 0.512948989868s real time; 0.51s CPU time 9 | stress_kalman(): 10 | Time spent: 0.541949987411s real time; 0.54s CPU time 11 | stress_kalman(): 12 | Time spent: 0.558674097061s real time; 0.55s CPU time 13 | stress_kalman(): 14 | Time spent: 0.589262008667s real time; 0.59s CPU time 15 | stress_kalman(): 16 | Time spent: 0.566255092621s real time; 0.56s CPU time 17 | stress_kalman(): 18 | Time spent: 0.539139032364s real time; 0.54s CPU time 19 | stress_kalman(): 20 | Time spent: 0.500986814499s real time; 0.5s CPU time 21 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/big_matlab.txt: -------------------------------------------------------------------------------- 1 | running kalman_stress... 2 | Success: 1 tests passed. 3 | Test time: 2.09 seconds. 4 | 5 | MATLAB: 0.459079 6 | MATLAB oo: 1.791310 7 | C++ 1: 0.500141 8 | C++ 2: 0.516933 9 | running kalman_stress... 10 | Success: 1 tests passed. 11 | Test time: 2.09 seconds. 12 | 13 | MATLAB: 0.385990 14 | MATLAB oo: 1.755816 15 | C++ 1: 0.520964 16 | C++ 2: 0.499602 17 | running kalman_stress... 18 | Success: 1 tests passed. 19 | Test time: 2.12 seconds. 20 | 21 | MATLAB: 0.425618 22 | MATLAB oo: 1.764192 23 | C++ 1: 0.504248 24 | C++ 2: 0.518150 25 | running kalman_stress... 26 | Success: 1 tests passed. 27 | Test time: 2.10 seconds. 28 | 29 | MATLAB: 0.417599 30 | MATLAB oo: 1.754509 31 | C++ 1: 0.552115 32 | C++ 2: 0.482225 33 | running kalman_stress... 34 | Success: 1 tests passed. 35 | Test time: 2.07 seconds. 36 | 37 | MATLAB: 0.445008 38 | MATLAB oo: 1.853825 39 | C++ 1: 0.486407 40 | C++ 2: 0.493320 41 | running kalman_stress... 42 | Success: 1 tests passed. 43 | Test time: 2.18 seconds. 44 | 45 | MATLAB: 0.401691 46 | MATLAB oo: 1.863537 47 | C++ 1: 0.579151 48 | C++ 2: 0.503747 49 | running kalman_stress... 50 | Success: 1 tests passed. 51 | Test time: 2.05 seconds. 52 | 53 | MATLAB: 0.435518 54 | MATLAB oo: 1.772984 55 | C++ 1: 0.483598 56 | C++ 2: 0.491486 57 | running kalman_stress... 58 | Success: 1 tests passed. 59 | Test time: 2.08 seconds. 60 | 61 | MATLAB: 0.397572 62 | MATLAB oo: 1.710642 63 | C++ 1: 0.520577 64 | C++ 2: 0.488352 65 | running kalman_stress... 66 | Success: 1 tests passed. 67 | Test time: 2.11 seconds. 68 | 69 | MATLAB: 0.418347 70 | MATLAB oo: 1.746707 71 | C++ 1: 0.530237 72 | C++ 2: 0.489408 73 | running kalman_stress... 74 | Success: 1 tests passed. 75 | Test time: 2.08 seconds. 76 | 77 | MATLAB: 0.456067 78 | MATLAB oo: 1.790138 79 | C++ 1: 0.507975 80 | C++ 2: 0.496146 81 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/big_python.txt: -------------------------------------------------------------------------------- 1 | stress_kalman(): 2 | Time spent: 0.675849914551s real time; 0.67s CPU time 3 | stress_kalman(): 4 | Time spent: 0.688468933105s real time; 0.69s CPU time 5 | stress_kalman(): 6 | Time spent: 0.679493904114s real time; 0.67s CPU time 7 | stress_kalman(): 8 | Time spent: 0.671465158463s real time; 0.67s CPU time 9 | stress_kalman(): 10 | Time spent: 0.680521011353s real time; 0.67s CPU time 11 | stress_kalman(): 12 | Time spent: 0.689363002777s real time; 0.69s CPU time 13 | stress_kalman(): 14 | Time spent: 0.69052696228s real time; 0.69s CPU time 15 | stress_kalman(): 16 | Time spent: 0.684376955032s real time; 0.68s CPU time 17 | stress_kalman(): 18 | Time spent: 0.686398983002s real time; 0.68s CPU time 19 | stress_kalman(): 20 | Time spent: 0.745839118958s real time; 0.75s CPU time 21 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/edgy-results.ods: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/benchmark_kalman/edgy-results.ods -------------------------------------------------------------------------------- /thesis/benchmark_kalman/huge_cython.txt: -------------------------------------------------------------------------------- 1 | stress_kalman(): 2 | Time spent: 1.76143193245s real time; 1.76s CPU time 3 | stress_kalman(): 4 | Time spent: 1.77156496048s real time; 1.76s CPU time 5 | stress_kalman(): 6 | Time spent: 1.78209090233s real time; 1.77s CPU time 7 | stress_kalman(): 8 | Time spent: 1.77805900574s real time; 1.77s CPU time 9 | stress_kalman(): 10 | Time spent: 1.84729385376s real time; 1.84s CPU time 11 | stress_kalman(): 12 | Time spent: 1.90192699432s real time; 1.88s CPU time 13 | stress_kalman(): 14 | Time spent: 1.81152796745s real time; 1.8s CPU time 15 | stress_kalman(): 16 | Time spent: 1.86418604851s real time; 1.86s CPU time 17 | stress_kalman(): 18 | Time spent: 1.79009199142s real time; 1.79s CPU time 19 | stress_kalman(): 20 | Time spent: 1.84711623192s real time; 1.84s CPU time 21 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/huge_matlab.txt: -------------------------------------------------------------------------------- 1 | running kalman_stress... 2 | Success: 1 tests passed. 3 | Test time: 9.43 seconds. 4 | 5 | MATLAB: 1.215931 6 | MATLAB oo: 3.636858 7 | C++ 1: 1.981949 8 | C++ 2: 1.951031 9 | running kalman_stress... 10 | Success: 1 tests passed. 11 | Test time: 9.33 seconds. 12 | 13 | MATLAB: 1.238566 14 | MATLAB oo: 3.534287 15 | C++ 1: 1.918337 16 | C++ 2: 1.955098 17 | running kalman_stress... 18 | Success: 1 tests passed. 19 | Test time: 9.37 seconds. 20 | 21 | MATLAB: 1.254767 22 | MATLAB oo: 3.683088 23 | C++ 1: 1.940426 24 | C++ 2: 1.938028 25 | running kalman_stress... 26 | Success: 1 tests passed. 27 | Test time: 9.22 seconds. 28 | 29 | MATLAB: 1.200529 30 | MATLAB oo: 4.170742 31 | C++ 1: 1.893287 32 | C++ 2: 1.891578 33 | running kalman_stress... 34 | Success: 1 tests passed. 35 | Test time: 9.54 seconds. 36 | 37 | MATLAB: 1.308060 38 | MATLAB oo: 4.648879 39 | C++ 1: 2.019786 40 | C++ 2: 1.966401 41 | running kalman_stress... 42 | Success: 1 tests passed. 43 | Test time: 9.54 seconds. 44 | 45 | MATLAB: 1.417822 46 | MATLAB oo: 4.194540 47 | C++ 1: 1.973873 48 | C++ 2: 2.003320 49 | running kalman_stress... 50 | Success: 1 tests passed. 51 | Test time: 9.46 seconds. 52 | 53 | MATLAB: 1.196948 54 | MATLAB oo: 3.587415 55 | C++ 1: 1.963522 56 | C++ 2: 1.969432 57 | running kalman_stress... 58 | Success: 1 tests passed. 59 | Test time: 9.20 seconds. 60 | 61 | MATLAB: 1.375206 62 | MATLAB oo: 3.831526 63 | C++ 1: 1.882542 64 | C++ 2: 1.896514 65 | running kalman_stress... 66 | Success: 1 tests passed. 67 | Test time: 9.42 seconds. 68 | 69 | MATLAB: 1.334835 70 | MATLAB oo: 3.417624 71 | C++ 1: 1.953324 72 | C++ 2: 1.954957 73 | running kalman_stress... 74 | Success: 1 tests passed. 75 | Test time: 9.43 seconds. 76 | 77 | MATLAB: 1.193737 78 | MATLAB oo: 3.783109 79 | C++ 1: 1.948582 80 | C++ 2: 1.968234 81 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/huge_python.txt: -------------------------------------------------------------------------------- 1 | stress_kalman(): 2 | Time spent: 2.08949303627s real time; 2.07s CPU time 3 | stress_kalman(): 4 | Time spent: 2.1090722084s real time; 2.11s CPU time 5 | stress_kalman(): 6 | Time spent: 2.13600301743s real time; 2.12s CPU time 7 | stress_kalman(): 8 | Time spent: 2.11966204643s real time; 2.11s CPU time 9 | stress_kalman(): 10 | Time spent: 2.10386610031s real time; 2.09s CPU time 11 | stress_kalman(): 12 | Time spent: 2.14292097092s real time; 2.14s CPU time 13 | stress_kalman(): 14 | Time spent: 2.15946602821s real time; 2.15s CPU time 15 | stress_kalman(): 16 | Time spent: 2.2303750515s real time; 2.23s CPU time 17 | stress_kalman(): 18 | Time spent: 2.05843496323s real time; 2.05s CPU time 19 | stress_kalman(): 20 | Time spent: 2.05605220795s real time; 2.05s CPU time 21 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/plot.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | import matplotlib.pyplot as plt 4 | 5 | x = [2, 30, 60] 6 | 7 | pybayes_py = [0.253716135, 0.6892303944, 2.1205345631] 8 | pybayes_cy = [0.0910650969, 0.5349771023, 1.8155289888] 9 | matlab_imper = [0.0690379, 0.4242489, 1.2736401] 10 | matlab_oo = [1.3781965, 1.780366, 3.8488068] 11 | bdm = [0.0255741, 0.5185413, 1.9475628] 12 | 13 | ax = plt.figure().add_subplot(111) 14 | plt.plot(x, pybayes_py, '--v', label='PyBayes Py') 15 | plt.plot(x, pybayes_cy, '-o', label='PyBayes Cy') 16 | plt.plot(x, matlab_imper, '-D', label='MATLAB imper.') 17 | plt.plot(x, matlab_oo, '--^', label='MATLAB o-o') 18 | plt.plot(x, bdm, '-s', label='BDM') 19 | 20 | plt.legend(loc='upper left') 21 | ax.set_xlabel('number of state-space dimensions') 22 | ax.set_ylabel('total run time [s]') 23 | ax.set_xlim(right=62) 24 | 25 | plt.show() 26 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/small_cython.txt: -------------------------------------------------------------------------------- 1 | stress_kalman(): 2 | Time spent: 0.095379114151s real time; 0.09s CPU time 3 | stress_kalman(): 4 | Time spent: 0.0871920585632s real time; 0.09s CPU time 5 | stress_kalman(): 6 | Time spent: 0.096510887146s real time; 0.09s CPU time 7 | stress_kalman(): 8 | Time spent: 0.0877079963684s real time; 0.09s CPU time 9 | stress_kalman(): 10 | Time spent: 0.0910830497742s real time; 0.09s CPU time 11 | stress_kalman(): 12 | Time spent: 0.0866940021515s real time; 0.09s CPU time 13 | stress_kalman(): 14 | Time spent: 0.0950100421906s real time; 0.1s CPU time 15 | stress_kalman(): 16 | Time spent: 0.0935218334198s real time; 0.1s CPU time 17 | stress_kalman(): 18 | Time spent: 0.086795091629s real time; 0.09s CPU time 19 | stress_kalman(): 20 | Time spent: 0.090756893158s real time; 0.1s CPU time 21 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/small_matlab.txt: -------------------------------------------------------------------------------- 1 | running kalman_stress... 2 | Success: 1 tests passed. 3 | Test time: 0.08 seconds. 4 | 5 | MATLAB: 0.071983 6 | MATLAB oo: 1.353003 7 | C++ 1: 0.024907 8 | C++ 2: 0.023958 9 | running kalman_stress... 10 | Success: 1 tests passed. 11 | Test time: 0.09 seconds. 12 | 13 | MATLAB: 0.077027 14 | MATLAB oo: 1.335515 15 | C++ 1: 0.026513 16 | C++ 2: 0.024623 17 | running kalman_stress... 18 | Success: 1 tests passed. 19 | Test time: 0.09 seconds. 20 | 21 | MATLAB: 0.069254 22 | MATLAB oo: 1.419338 23 | C++ 1: 0.026283 24 | C++ 2: 0.024899 25 | running kalman_stress... 26 | Success: 1 tests passed. 27 | Test time: 0.08 seconds. 28 | 29 | MATLAB: 0.070209 30 | MATLAB oo: 1.384254 31 | C++ 1: 0.023901 32 | C++ 2: 0.024420 33 | running kalman_stress... 34 | Success: 1 tests passed. 35 | Test time: 0.08 seconds. 36 | 37 | MATLAB: 0.063388 38 | MATLAB oo: 1.424014 39 | C++ 1: 0.024526 40 | C++ 2: 0.023648 41 | running kalman_stress... 42 | Success: 1 tests passed. 43 | Test time: 0.08 seconds. 44 | 45 | MATLAB: 0.063648 46 | MATLAB oo: 1.323431 47 | C++ 1: 0.023472 48 | C++ 2: 0.023305 49 | running kalman_stress... 50 | Success: 1 tests passed. 51 | Test time: 0.09 seconds. 52 | 53 | MATLAB: 0.062904 54 | MATLAB oo: 1.317247 55 | C++ 1: 0.030537 56 | C++ 2: 0.024352 57 | running kalman_stress... 58 | Success: 1 tests passed. 59 | Test time: 0.09 seconds. 60 | 61 | MATLAB: 0.062819 62 | MATLAB oo: 1.373581 63 | C++ 1: 0.024858 64 | C++ 2: 0.023536 65 | running kalman_stress... 66 | Success: 1 tests passed. 67 | Test time: 0.09 seconds. 68 | 69 | MATLAB: 0.078756 70 | MATLAB oo: 1.502431 71 | C++ 1: 0.028118 72 | C++ 2: 0.024335 73 | running kalman_stress... 74 | Success: 1 tests passed. 75 | Test time: 0.08 seconds. 76 | 77 | MATLAB: 0.070391 78 | MATLAB oo: 1.349151 79 | C++ 1: 0.022626 80 | C++ 2: 0.02249 81 | -------------------------------------------------------------------------------- /thesis/benchmark_kalman/small_python.txt: -------------------------------------------------------------------------------- 1 | stress_kalman(): 2 | Time spent: 0.246462106705s real time; 0.25s CPU time 3 | stress_kalman(): 4 | Time spent: 0.2493019104s real time; 0.25s CPU time 5 | stress_kalman(): 6 | Time spent: 0.268669843674s real time; 0.26s CPU time 7 | stress_kalman(): 8 | Time spent: 0.244338989258s real time; 0.24s CPU time 9 | stress_kalman(): 10 | Time spent: 0.246606111526s real time; 0.24s CPU time 11 | stress_kalman(): 12 | Time spent: 0.244695901871s real time; 0.25s CPU time 13 | stress_kalman(): 14 | Time spent: 0.243955135345s real time; 0.25s CPU time 15 | stress_kalman(): 16 | Time spent: 0.274691104889s real time; 0.27s CPU time 17 | stress_kalman(): 18 | Time spent: 0.258280038834s real time; 0.26s CPU time 19 | stress_kalman(): 20 | Time spent: 0.260160207748s real time; 0.27s CPU time 21 | -------------------------------------------------------------------------------- /thesis/benchmark_py_numpy/spectralnorm_numpy.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # The Computer Language Benchmarks Game 3 | # http://shootout.alioth.debian.org/ 4 | # 5 | # Contributed by Sebastien Loisel 6 | # Fixed by Isaac Gouy 7 | # Sped up by Josh Goldfoot 8 | # Dirtily sped up by Simon Descarpentries 9 | # Sped up with numpy by Kittipong Piyawanno 10 | # 2to3 11 | 12 | from sys import argv 13 | from numpy import * 14 | 15 | def spectralnorm(n): 16 | u = matrix(ones(n)) 17 | j = arange(n) 18 | eval_func = lambda i : 1.0 / ((i + j) * (i + j + 1) / 2 + i + 1) 19 | M = matrix([eval_func(i) for i in arange(n)]) 20 | MT = M.T 21 | for i in range (10): 22 | v = (u*MT)*M 23 | u = (v*MT)*M 24 | print("%0.9f" % (sum(u*v.T)/sum(v*v.T))**0.5) 25 | 26 | spectralnorm(int(argv[1])) 27 | -------------------------------------------------------------------------------- /thesis/benchmark_py_numpy/spectralnorm_plain_1thread.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # The Computer Language Benchmarks Game 3 | # http://shootout.alioth.debian.org/ 4 | # Contributed by Sebastien Loisel 5 | # Fixed by Isaac Gouy 6 | # Sped up by Josh Goldfoot 7 | # Dirtily sped up by Simon Descarpentries 8 | # Concurrency by Jason Stitt 9 | # 2to3 10 | 11 | from multiprocessing import Pool 12 | from math import sqrt 13 | 14 | from sys import argv 15 | 16 | def eval_A (i, j): 17 | return 1.0 / ((i + j) * (i + j + 1) / 2 + i + 1) 18 | 19 | def eval_A_times_u (u): 20 | args = ((i,u) for i in range(len(u))) 21 | return pool.map(part_A_times_u, args) 22 | 23 | def eval_At_times_u (u): 24 | args = ((i,u) for i in range(len(u))) 25 | return pool.map(part_At_times_u, args) 26 | 27 | def eval_AtA_times_u (u): 28 | return eval_At_times_u (eval_A_times_u (u)) 29 | 30 | def part_A_times_u(xxx_todo_changeme): 31 | (i,u) = xxx_todo_changeme 32 | partial_sum = 0 33 | for j, u_j in enumerate(u): 34 | partial_sum += eval_A (i, j) * u_j 35 | return partial_sum 36 | 37 | def part_At_times_u(xxx_todo_changeme1): 38 | (i,u) = xxx_todo_changeme1 39 | partial_sum = 0 40 | for j, u_j in enumerate(u): 41 | partial_sum += eval_A (j, i) * u_j 42 | return partial_sum 43 | 44 | def main(): 45 | n = int(argv[1]) 46 | u = [1] * n 47 | 48 | for dummy in range (10): 49 | v = eval_AtA_times_u (u) 50 | u = eval_AtA_times_u (v) 51 | 52 | vBv = vv = 0 53 | 54 | for ue, ve in zip (u, v): 55 | vBv += ue * ve 56 | vv += ve * ve 57 | 58 | print("%0.9f" % (sqrt(vBv/vv))) 59 | 60 | if __name__ == '__main__': 61 | pool = Pool(processes=1) 62 | main() 63 | -------------------------------------------------------------------------------- /thesis/benchmark_py_numpy/spectralnorm_plain_4threads.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # The Computer Language Benchmarks Game 3 | # http://shootout.alioth.debian.org/ 4 | # Contributed by Sebastien Loisel 5 | # Fixed by Isaac Gouy 6 | # Sped up by Josh Goldfoot 7 | # Dirtily sped up by Simon Descarpentries 8 | # Concurrency by Jason Stitt 9 | # 2to3 10 | 11 | from multiprocessing import Pool 12 | from math import sqrt 13 | 14 | from sys import argv 15 | 16 | def eval_A (i, j): 17 | return 1.0 / ((i + j) * (i + j + 1) / 2 + i + 1) 18 | 19 | def eval_A_times_u (u): 20 | args = ((i,u) for i in range(len(u))) 21 | return pool.map(part_A_times_u, args) 22 | 23 | def eval_At_times_u (u): 24 | args = ((i,u) for i in range(len(u))) 25 | return pool.map(part_At_times_u, args) 26 | 27 | def eval_AtA_times_u (u): 28 | return eval_At_times_u (eval_A_times_u (u)) 29 | 30 | def part_A_times_u(xxx_todo_changeme): 31 | (i,u) = xxx_todo_changeme 32 | partial_sum = 0 33 | for j, u_j in enumerate(u): 34 | partial_sum += eval_A (i, j) * u_j 35 | return partial_sum 36 | 37 | def part_At_times_u(xxx_todo_changeme1): 38 | (i,u) = xxx_todo_changeme1 39 | partial_sum = 0 40 | for j, u_j in enumerate(u): 41 | partial_sum += eval_A (j, i) * u_j 42 | return partial_sum 43 | 44 | def main(): 45 | n = int(argv[1]) 46 | u = [1] * n 47 | 48 | for dummy in range (10): 49 | v = eval_AtA_times_u (u) 50 | u = eval_AtA_times_u (v) 51 | 52 | vBv = vv = 0 53 | 54 | for ue, ve in zip (u, v): 55 | vBv += ue * ve 56 | vv += ve * ve 57 | 58 | print("%0.9f" % (sqrt(vBv/vv))) 59 | 60 | if __name__ == '__main__': 61 | pool = Pool(processes=4) 62 | main() 63 | -------------------------------------------------------------------------------- /thesis/bibliography.bib: -------------------------------------------------------------------------------- 1 | @Book{ThrBurFox:05, 2 | title = {{Probabilistic Robotics (Intelligent Robotics and Autonomous Agents)}}, 3 | author = {Thrun, S. and Burgard, W. and Fox, D.}, 4 | year = {2005}, 5 | publishe = {MIT press, Cambridge, Massachusetts, USA} 6 | } 7 | @Article{Gus:02, 8 | title = {{Particle filters for positioning, navigation, and tracking}}, 9 | author = {Gustafsson, F. and Gunnarsson, F. and Bergman, N. and Forssell, U. and Jansson, J. and 10 | Karlsson, R. and Nordlund, P.J.}, 11 | journal = {Signal Processing, IEEE Transactions on}, 12 | volume = {50}, 13 | number = {2}, 14 | pages = {425--437}, 15 | issn = {1053-587X}, 16 | year = {2002}, 17 | publisher = {IEEE} 18 | } 19 | @Article{HofSmi:09, 20 | author = {Hofman, R. and \v{S}m\'\i{}dl V.}, 21 | title = {Assimilation of spatio-temporal distribution of radionuclides in early phase of 22 | radiation accident}, 23 | journal = {Bezpečnost jaderné energie}, 24 | volume = 18, 25 | pages = {226--228}, 26 | year = {2010}, 27 | } 28 | @InProceedings{HofSmiPech:09, 29 | author= {Hofman, R. and \v{S}m\'\i{}dl, V. and Pecha, P.}, 30 | title = {Data assimilation in early phase of radiation accident using particle filter}, 31 | booktitle ={The Fifth WMO International Symposium on Data Assimilation}, 32 | address = {Melbourne, Australia}, 33 | year = {2009}, 34 | } 35 | @InProceedings{PechHofSmi:09, 36 | author = {Pecha, P. and Hofman R. and \v{S}m\'\i{}dl, V.}, 37 | title = {Bayesian tracking of the toxic plume spreading in the early stage of radiation accident}, 38 | booktitle = {Proceedings of the 2009 European Simulation and Modelling Conference}, 39 | address = {Leicester, GB}, 40 | year = 2009, 41 | } 42 | 43 | @Article{AruMasGor:02, 44 | author = {M. Sanjeev Arulampalam and Simon Maskell and Neil Gordon}, 45 | title = {A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking}, 46 | journal = {IEEE Transactions on Signal Processing}, 47 | year = {2002}, 48 | volume = {50}, 49 | pages = {174--188} 50 | } 51 | @InCollection{Pet:81, 52 | author = {Peterka, V.}, 53 | title = {Bayesian Approach to System Identification}, 54 | booktitle = {Trends and Progress in System identification}, 55 | publisher = {Pergamon Press}, 56 | year = {1981}, 57 | editor = {P. Eykhoff}, 58 | pages = {239--304}, 59 | address = {Oxford} 60 | } 61 | @Article{CriDou:02, 62 | title={A survey of convergence results on particle filtering methods for practitioners}, 63 | author={Crisan, D. and Doucet, A.}, 64 | journal={Signal Processing, IEEE Transactions on}, 65 | volume={50}, 66 | number={3}, 67 | pages={736--746}, 68 | year={2002}, 69 | publisher={IEEE} 70 | } 71 | @Article{SchGusNor:05, 72 | author={Schön, T. B. and Gustafsson, F. and Nordlund, P.-J.}, 73 | journal={Signal Processing, IEEE Transactions on}, 74 | title={Marginalized particle filters for mixed linear/nonlinear state-space models}, 75 | year={2005}, 76 | month={july}, 77 | volume={53}, 78 | number={7}, 79 | pages={2279--2289}, 80 | doi={10.1109/TSP.2005.849151}, 81 | ISSN={1053-587X} 82 | } 83 | @Misc{SchKarGus:06, 84 | author = {Schön, T. B. and Karlsson, R. and Gustafsson, F.}, 85 | title = {The Marginalized Particle Filter --- Analysis, Applications and Generalizations}, 86 | year = {2006} 87 | } 88 | 89 | @InProceedings{Smi:10, 90 | author={{\v{S}}m{\'{\i}}dl, V.}, 91 | title={Software Analysis Unifying Particle Filtering and Marginalized Particle Filtering}, 92 | booktitle={Proceedings of the 13th International Conference on Information Fusion}, 93 | publisher={IET}, 94 | location={Edinburgh}, 95 | country={GB}, 96 | year={2010}, 97 | } 98 | @PhdThesis{Smi:05, 99 | author = {\v{S}m\'{i}dl, V.}, 100 | title = {Software analysis of Bayesian distributed dynamic decision making}, 101 | year = {2005}, 102 | address = {Plze\v{n}}, 103 | school = {University of West Bohemia, Faculty of Applied Sciences, Pilsen, Czech Republic}, 104 | url = {http://www.kky.zcu.cz/en/publications/SmidlV_2005_Softwareanalysisof}, 105 | } 106 | 107 | @Article{LevPet:06, 108 | author = {Levanoni, Y. and Petrank, E.}, 109 | title = {An On-the-Fly Reference Counting Garbage Collector for {J}ava}, 110 | journal = {ACM Transactions on Programming Languages and Systems}, 111 | volume = {28}, 112 | number = {1}, 113 | month = {January}, 114 | year = {2006}, 115 | url = {http://portal.acm.org/ft_gateway.cfm?id=1111597&type=pdf&coll=portal&dl=ACM&CFID=72488985&CFTOKEN=2165367} 116 | } 117 | @Book{Str:00, 118 | title = {The C++ programming language, Third Edition}, 119 | author = {Stroustrup, B.}, 120 | isbn = {9780201700732}, 121 | lccn = {99059344}, 122 | year = {2000}, 123 | publisher = {Addison-Wesley} 124 | } 125 | @Misc{BFL, 126 | author = {Gadeyne, K.}, 127 | title = {{BFL}: {B}ayesian {F}iltering {L}ibrary}, 128 | year = {2001}, 129 | howpublished = {\url{http://www.orocos.org/bfl}}, 130 | } 131 | @InProceedings{BDM, 132 | author = {Šmídl, V. and Pištěk, M.}, 133 | title = {Presentation of {B}ayesian {D}ecision {M}aking {T}oolbox ({BDM})}, 134 | booktitle = {Abstracts of Contributions to 5th International Workshop on 135 | Data --- Algorithms --- Decision Making}, 136 | pages = {37}, 137 | address = {Praha}, 138 | year = {2009}, 139 | editor = {Janžura, M. and Ivánek, J.}, 140 | } 141 | 142 | @InProceedings{BehBraSel:09, 143 | author = {Stefan Behnel and Robert W. Bradshaw and Dag Sverre Seljebotn}, 144 | title = {Cython tutorial}, 145 | booktitle = {Proceedings of the 8th Python in Science Conference}, 146 | pages = {4--14}, 147 | address = {Pasadena, CA USA}, 148 | year = {2009}, 149 | editor = {Ga\"el Varoquaux and St\'efan van der Walt and Jarrod Millman}, 150 | howpublished = "\url{http://conference.scipy.org/proceedings/SciPy2009/paper_1}", 151 | } 152 | @InProceedings{Sel:09, 153 | author = {Dag Sverre Seljebotn}, 154 | title = {Fast numerical computations with Cython}, 155 | booktitle = {Proceedings of the 8th Python in Science Conference}, 156 | pages = {15--22}, 157 | address = {Pasadena, CA USA}, 158 | year = {2009}, 159 | editor = {Ga\"el Varoquaux and St\'efan van der Walt and Jarrod Millman}, 160 | howpublished = "\url{http://conference.scipy.org/proceedings/SciPy2009/paper_2}", 161 | } 162 | @Article{BehBraCitDalSelSmi:11, 163 | author={Behnel, S. and Bradshaw, R. and Citro, C. and Dalcin, L. and Seljebotn, D.S. and Smith, K.}, 164 | journal={Computing in Science Engineering}, 165 | title={Cython: The Best of Both Worlds}, 166 | year={2011}, 167 | month={march-april}, 168 | volume={13}, 169 | number={2}, 170 | pages={31--39}, 171 | keywords={Cython language;Fortran code;Python language extension;numerical loops;programming language;C language;numerical analysis;}, 172 | doi={10.1109/MCSE.2010.118}, 173 | ISSN={1521-9615}, 174 | } 175 | -------------------------------------------------------------------------------- /thesis/chap0-introduction.tex: -------------------------------------------------------------------------------- 1 | \clearpage % so that table of contents mentions correct page 2 | \phantomsection % so that hyperref makes correct reference 3 | \addcontentsline{toc}{chapter}{Introduction} 4 | \pagenumbering{arabic} 5 | \chapter*{Introduction} 6 | 7 | Bayesian filtering (or, recursive Bayesian estimation) is a very promising approach to estimation 8 | of dynamic systems; it can be applied to a wide range of real-world problems in areas such as 9 | robotics~\cite{ThrBurFox:05,Gus:02} (tracking, navigation), environmental simulations --- 10 | e.g. tracking of the radioactive plume upon nuclear 11 | accident~\cite{HofSmi:09,HofSmiPech:09,PechHofSmi:09}, econometrics and many more. 12 | 13 | While many Bayesian filtering algorithms are simple enough to be implemented in software on 14 | \emph{ad-hoc} basis, it is proposed that a well designed library can bring many advantages such as 15 | ability to combine and interchange individual methods, better performance, programmer convenience 16 | and a tested code-base. 17 | 18 | \noindent{}The text is organised as follows: 19 | \begin{enumerate} 20 | \item Theoretical background of Bayesian filtering is presented in the first chapter along with 21 | a description of well-known Bayesian filters: the Kalman filter, the particle filter and a 22 | simple form of the marginalized particle filter. 23 | \item In chapter 2 a software analysis for a desired library for Bayesian filtering is 24 | performed: requirements are set up, general approaches to software development are discussed and 25 | programming languages C++, MATLAB and Python and their implementations are compared. Interesting 26 | results are achieved when Python is combined with Cython. 27 | \item The PyBayes library that was developed as a part of this thesis is presented. PyBayes is 28 | written in Python with an option to use Cython and implements all algorithms presented in the 29 | first chapter. Performance of PyBayes is measured and confronted with concurrent implementations 30 | that use different implementation environments. 31 | \end{enumerate} 32 | Bayesian filtering is a subtask of a broader topic of Bayesian decision-making~\cite{Smi:05}; while 33 | decision-making is not covered in this text, we expect the PyBayes library to form a good building 34 | block for implementing decision-making systems. 35 | -------------------------------------------------------------------------------- /thesis/chap0-notation.tex: -------------------------------------------------------------------------------- 1 | \chapter*{Notation} \addcontentsline{toc}{chapter}{Notation} 2 | 3 | Following notation is used throughout this text: 4 | 5 | \bigskip 6 | 7 | \begin{tabular}{l p{0.8\textwidth}} 8 | \(\mathbb{N}\) & set of natural numbers excluding zero \\ 9 | \(\mathbb{N}_0\) & set of natural numbers including zero \\ 10 | \(\mathbb{R}\) & set of real numbers \\ 11 | \(t\) & discrete time moment; \(t \in \mathbb{N}_0\) \\ 12 | \(a_t\) & value of quantity \(a\) at time \(t\); \(a_t \in \mathbb{R}^n, n \in \mathbb{N}\) \\ 13 | \(a_{t|t'}\) & quantity with two indices: \(t\) and \(t'\) \\ 14 | & there is no implicit link between \(a_{t}\), \(a_{t|t'}\) and \(a_{t'}\) \\ 15 | \(a_{t:t'}\) & sequence of quantities \((a_t, a_{t+1}, \dotsc, a_{t'-1}, a_{t'})\) \\ 16 | \(p(a_t)\) & probability density function{\footnotemark[1]} of quantity \(a\) at time \(t\) (unless 17 | noted otherwise) \\ 18 | \(p(a_t|b_{t'})\) & conditional {\pdf} of quantity \(a\) at time \(t\) given value of quantity 19 | \(b\) at time \(t'\) \\ 20 | \(\delta(a)\) & Dirac delta function; used exclusively in context of {\pdfs} to denote discrete 21 | distribution within framework of continuous distributions{\footnotemark[2]} \\ 22 | \(\mathcal{N}(\mu, \Sigma)\) & multivariate normal (Gaussian) {\pdf} with mean vector \(\mu\) 23 | and covariance matrix \(\Sigma\) \\ 24 | \end{tabular} 25 | 26 | % \footnote does not work in tabular environment, - this can be worked around with this 27 | \footnotetext[1]{for the purpose of this text, {\pdf} \(p\) is multivariate non-negative function 28 | \(\mathbb{R}^n \rightarrow \mathbb{R}; \; \int_{\supp p} p(x_1, x_2, \dotsc, x_n) \; \dx_1 \dx_2 \dotsb \dx_n = 1\)} 29 | 30 | \footnotetext[2]{so that \(\int_{-\infty}^{\infty} f(x) \delta(x - \mu) \; \dx = f(\mu)\) and more 31 | complex expressions can be derived using integral linearity and Fubini's theorem.} 32 | -------------------------------------------------------------------------------- /thesis/chap1-theory.tex: -------------------------------------------------------------------------------- 1 | \chapter{Basics of Recursive Bayesian Estimation} 2 | 3 | In following sections the problem of recursive Bayesian estimation (Bayesian filtering) is 4 | stated and its analytical solution is derived. Later on, due to practical intractability of the 5 | solution in its general form, a few methods that either simplify the problem or approximate the 6 | solution are shown. 7 | 8 | \section{Problem Statement} 9 | 10 | Assume a dynamic system described by a hidden real-valued \emph{state vector} \(x\) which evolves at 11 | discrete time steps according to a known function \(f_t\) (in this text called \emph{process model}) 12 | as described by \eqref{eq:DynSysFt}. 13 | \begin{equation} \label{eq:DynSysFt} 14 | x_t = f_t(x_{t-1}, v_{t-1}) 15 | \end{equation} 16 | 17 | Variable \(v_t\) in \eqref{eq:DynSysFt} denotes random \emph{process noise}, which may come from various 18 | sources and is often inevitable. Sequence of \(v_t\) is assumed to be identically independently 19 | distributed random variable sequence. 20 | 21 | The state of the system is hidden and can only be observed though a real-valued \emph{observation vector} 22 | \(y\) that relates to the state \(x\) as in \eqref{eq:DynSysHt}, but adds further \emph{observation 23 | noise} \(w\). 24 | \begin{equation} \label{eq:DynSysHt} 25 | y_t = h_t(x_t, w_t) 26 | \end{equation} 27 | 28 | In \eqref{eq:DynSysHt} \(h_t\) is known function called \emph{observation model} in this text and \(w_t\) is 29 | identically independently distributed random variable sequence that denotes observation noise. 30 | 31 | The goal of recursive\footnote{by the word recursive we mean that it is not needed to keep track of 32 | the whole batch of previous observations in practical methods, only appropriate quantities from time 33 | moments \(t-1\) and \(t\) are needed to estimate \(x_t\). However, this does not apply to the 34 | derivation of the solution, where the notation of whole batch of observations \(y_{1:t}\) is used.} 35 | Bayesian estimation is to give an estimate of the state \(x_t\) given the 36 | observations \(y_{1:t}\) provided the knowledge of the functions \(f_t\) and \(h_t\). 37 | More formally, the goal is to find the {\pdf} \(p(x_t | y_{1:t})\). 38 | Theoretical solution to this problem is known and is presented in next section. 39 | 40 | \section{Theoretical solution} 41 | 42 | At first, we observe that {\pdf} \(p(x_t|x_{t-1})\) can be derived from the process model 43 | \eqref{eq:DynSysFt} (given the distribution of \(v_k\)) and that \(p(y_t|x_t)\) can be derived from 44 | the observation model \eqref{eq:DynSysHt} respectively. (given the distribution of \(w_k\)) 45 | 46 | Because recursive solution is requested, suppose that \(p(x_{t-1}|y_{1:t-1})\) and 47 | \(p(x_0)\) are known\footnote{\(p(x_0)\) can be called initial {\pdf} of the state vector.} in 48 | order to be able to make the transition \(t-1 \; \rightarrow \; t\). 49 | 50 | In the first stage that can be called \emph{prediction}, \emph{prior} {\pdf} 51 | \(p(x_t | y_{1:t-1})\) is calculated without knowledge of \(y_t\). We begin the derivation by 52 | performing the reverse of the marginalization over \(x_{k-1}\). 53 | \begin{equation*} 54 | p(x_t | y_{1:t-1}) = \int_{-\infty}^{\infty} p(x_t, x_{t-1} | y_{1:t-1}) \; \dx_{t-1} 55 | \end{equation*} 56 | 57 | Using chain rule for {\pdfs}, the element of integration can be split. 58 | \begin{equation*} 59 | p(x_t | y_{1:t-1}) = \int_{-\infty}^{\infty} p(x_t | x_{t-1}, y_{1:t-1}) p(x_{t-1} | y_{1:t-1}) \; \dx_{t-1} 60 | \end{equation*} 61 | 62 | With an assumption that the modelled dynamic system \eqref{eq:DynSysFt} possesses \emph{Markov 63 | Property}\footnote{an assumption of independence that states that system state in time \(t\) only 64 | depends on system state in \(t-1\) (and is not directly affected by previous states).}, 65 | \(p(x_t | x_{t-1}, y_{1:t-1})\) equals \(p(x_t | x_{t-1})\).~\cite{AruMasGor:02} 66 | This leaves us with the result \eqref{eq:PriorPdf}. 67 | \begin{equation} \label{eq:PriorPdf} 68 | p(x_t | y_{1:t-1}) = \int_{-\infty}^{\infty} p(x_t | x_{t-1}) p(x_{t-1} | y_{1:t-1}) \; \dx_{t-1} 69 | \end{equation} 70 | 71 | As we can see, prior {\pdf} only depends on previously known functions and therefore can be 72 | calculated. 73 | 74 | We continue with the second stage that could be named \emph{update}, where new observation \(y_t\) is taken into 75 | account and \emph{posterior} {\pdf} \(p(x_t | y_{1:t})\) is calculated. Bayes' theorem can be used 76 | to derive posterior {\pdf} \eqref{eq:PosteriorPdfRaw}. 77 | \begin{equation} \label{eq:PosteriorPdfRaw} 78 | p(x_t | y_{1:t}) = \frac{p(y_t | x_t, y_{1:t-1}) p(x_t | y_{1:t-1})}{p(y_t | y_{1:t-1})} 79 | \end{equation} 80 | 81 | According to the observation model \eqref{eq:DynSysHt} and assuming Markov property, \(y_t\) only 82 | depends on \(x_t\). That is \(p(y_t | x_t, y_{1:t-1}) = p(y_t | x_t)\). Therefore posterior 83 | {\pdf} can be further simplified into \eqref{eq:PosteriorPdf}. 84 | \begin{equation} \label{eq:PosteriorPdf} 85 | p(x_t | y_{1:t}) = \frac{p(y_t | x_t) p(x_t | y_{1:t-1})}{p(y_t | y_{1:t-1})} 86 | \end{equation} 87 | 88 | While both {\pdfs} in the numerator of \eqref{eq:PosteriorPdf} are already known, \(p(y_t|y_{1:t-1})\) 89 | found in the denominator can be calculated using the formula \eqref{eq:MargLikelihood}, where 90 | marginalization over \(x_t\) is preformed. Quantity \eqref{eq:MargLikelihood} can also be interpreted as 91 | \emph{marginal likelihood} (sometimes called \emph{evidence}) of observation.~\cite{Smi:10} 92 | \begin{equation} \label{eq:MargLikelihood} 93 | p(y_t | y_{1:t-1}) = \int_{-\infty}^{\infty} p(y_t | x_t) p(x_t | y_{1:t-1}) \; \dx_{t} 94 | \end{equation} 95 | 96 | Computing \eqref{eq:MargLikelihood} isn't however strictly needed as it does not depend on \(x_t\) and 97 | serves as a normalising constant in \eqref{eq:PosteriorPdf}. Depending on use-case the normalising 98 | constant may not be needed at all or may be computed alternatively using the fact that \(p(x_t | y_{1:y})\) 99 | integrates to \(1\). 100 | 101 | We have shown that so called \emph{optimal Bayesian solution}\cite{AruMasGor:02} can be easily 102 | analytically inferred using only \emph{chain rule for {\pdfs}}, \emph{marginalization} and 103 | \emph{Bayes' theorem}. (equations \eqref{eq:PriorPdf}, \eqref{eq:PosteriorPdf} and 104 | \eqref{eq:MargLikelihood} forming the main steps of the solution) On the other hand, using this 105 | method directly in practice proves difficult because at least one parametric multidimensional 106 | integration has to be performed (in \eqref{eq:PriorPdf}), which is (in its general form) hardly 107 | tractable for greater than small state vector dimensions. 108 | 109 | This is a motivation for various simplifications and approximations among which we have chosen 110 | a Kalman filter described in the next section and a family of particle filters described later. 111 | 112 | \section{Kalman Filter} 113 | 114 | The Kalman filter\footnote{first presented by Rudolf Emil Kalman in 1960.} poses additional set of strong 115 | assumptions on modelled dynamic system, but greatly 116 | simplifies the optimal Bayesian solution \eqref{eq:PriorPdf}, \eqref{eq:PosteriorPdf} into a 117 | sequence of algebraic operations with matrices. On the other hand, when these requirements can be 118 | fulfilled, there is no better estimator in the Bayesian point of view because the Kalman filter computes 119 | \(p(x_t | y_{1:t})\) \emph{exactly.}\footnote{not accounting for numeric errors that arise in 120 | practical implementations.} 121 | 122 | Assumptions additionally posed on system by the the Kalman filter are: 123 | \begin{enumerate} 124 | \item \(f_t\) in the process model \eqref{eq:DynSysFt} is a linear function of \(x_t\) and 125 | \(v_t\). 126 | \item \(v_t \sim \mathcal{N}(0, Q_t)\) meaning that process noise \(v_t\) is normally 127 | distributed with zero mean\footnote{zero mean assumption is not strictly needed, it is however 128 | common in many implementations.} and with known covariance matrix \(Q_t\). 129 | \item \(h_t\) in the observation model \eqref{eq:DynSysHt} is a linear function of \(x_t\) and 130 | \(w_t\). 131 | \item \(w_t \sim \mathcal{N}(0, R_t)\) meaning that observation noise \(w_t\) is normally distributed 132 | with zero mean and with known covariance matrix \(R_t\). 133 | \item initial state {\pdf} is Gaussian. 134 | \end{enumerate} 135 | 136 | It can be proved that if the above assumptions hold, \(p(x_t|y_{1:t})\) is Gaussian for all 137 | \(t > 0\).~\cite{Pet:81} Furthermore, given assumptions 1. and 2. the process model 138 | \eqref{eq:DynSysFt} can be reformulated as \eqref{eq:LinSysAt}, where \(A_t\) is real-valued matrix 139 | that represents \(f_t\). 140 | Using the same idea and assumptions 3. and 4. the observation model \eqref{eq:DynSysHt} can be 141 | expressed as \eqref{eq:LinSysCt}, \(C_t\) being real-valued matrix representing \(h_t\). Another 142 | common requirement used below in the algorithm description is that \(v_t\) and \(w_t\) are 143 | stochastically independent. 144 | \begin{align} 145 | x_t &= A_t x_{t-1} + \hat{v}_{t-1} & A_t &\in \mathbb{R}^{n,n} \;\; n \in \mathbb{N} \label{eq:LinSysAt} \\ 146 | y_t &= C_t x_t + \hat{w}_t & C_t &\in \mathbb{R}^{j,n} \;\; j \in \mathbb{N} \;\; j \leq n \label{eq:LinSysCt} 147 | \end{align} 148 | 149 | Note that we have marked the noises \(v_t\) and \(w_t\) as \(\hat{v}_t\) and \(\hat{w}_t\) when they 150 | are transformed through \(A_t\), respectively \(C_t\) matrix. Let also \(\hat{Q}_t\) denote the 151 | covariance matrix of \(\hat{v}_t\) and \(\hat{R}_t\) denote the covariance matrix of \(\hat{w}_t\) 152 | in further text. 153 | 154 | At this point we can describe the algorithm of the Kalman filter. As stated above, posterior {\pdf} 155 | is Gaussian and thus can be parametrised by mean vector \(\mu\) and covariance matrix \(P\). Let us 156 | denote posterior mean from previous iteration by \(\mu_{t-1|t-1}\) and associated covariance by 157 | \(P_{t-1|t-1}\) as in \eqref{eq:KalmanPreAPost}. 158 | \begin{equation} \label{eq:KalmanPreAPost} 159 | p(x_{t-1} | y_{1:t-1}) = \mathcal{N}(\mu_{t-1|t-1}, P_{t-1|t-1}) 160 | \end{equation} 161 | 162 | Prior {\pdf} \eqref{eq:KalmanAPrior} can then be calculated as follows:~\cite{AruMasGor:02} 163 | \begin{align} 164 | p(x_t | y_{1:t-1}) &= \mathcal{N}(\mu_{t|t-1}, P_{t|t-1}) \label{eq:KalmanAPrior} \\ 165 | \mu_{t|t-1} &= A_t \mu_{t-1|t-1} \notag \\ 166 | P_{t|t-1} &= A_t P_{t-1|t-1} A_t^T + \hat{Q}_{t-1} \notag 167 | \end{align} 168 | 169 | Before introducing posterior {\pdf} it is useful to establish another Gaussian {\pdf} 170 | \eqref{eq:KalmanEvidence} that is not necessarily needed, but is useful because it represents 171 | marginal likelihood \eqref{eq:MargLikelihood}. % TODO: citation for this! 172 | \begin{align} 173 | p(y_t|y_{1:t-1}) &= \mathcal{N}(\nu_{t|t-1}, S_{t|t-1}) \label{eq:KalmanEvidence} \\ 174 | \nu_{t|t-1} &= C_t \mu_{t|t-1} \notag \\ 175 | S_{t|t-1} &= C_t P_{t|t-1} C_t^T + \hat{R}_t \notag 176 | \end{align} 177 | 178 | The update phase of the Kalman filter can be performed by computing so-called \emph{Kalman gain} matrix 179 | \eqref{eq:KalmanGain}, posterior {\pdf} \eqref{eq:KalmanAPost} is then derived from prior one 180 | using the Kalman gain \(K_t\) and observation \(y_t\).~\cite{AruMasGor:02} 181 | \begin{align} 182 | K_t &= P_{t|t-1} C_t^T S_{t|t-1}^{-1} \label{eq:KalmanGain} \\[\parskip] 183 | p(x_t|y_{1:t}) &= \mathcal{N}(\mu_{t|t}, P_{t|t}) \label{eq:KalmanAPost} \\ 184 | \mu_{t|t} &= \mu_{t|t-1} + K_t(y_t - \nu_{t|t-1}) \notag \\ 185 | P_{t|t} &= P_{t|t-1} - K_t C_t P_{t|t-1} \notag 186 | \end{align} 187 | 188 | In all formulas above \(A^T\) denotes a transpose of matrix \(A\) and \(A^{-1}\) denotes inverse 189 | matrix to \(A\). As can be seen, formulas \eqref{eq:PriorPdf} and \eqref{eq:PosteriorPdf} have 190 | been reduced to tractable algebraic operations, computing inverse matrix\footnote{it can be shown 191 | that \(S_{t|t-1}\) is positive definite given that \(C_t\) is full-ranked, 192 | therefore the inverse in \eqref{eq:KalmanGain} exists.} being the most costly one. 193 | 194 | It should be further noted that the Kalman filter and described algorithm can be easily enhanced to 195 | additionally cope with an \emph{intervention} (or control) vector applied to the system, making it 196 | suitable for the theory of decision-making. Numerous generalisations of the Kalman filter exist, for 197 | example an \emph{extended Kalman filter} that relaxes the requirement of the linear system by locally 198 | approximating a non-linear system with Taylor series. These are out of scope of this 199 | text, but provide areas for subsequent consideration. 200 | 201 | On the other hand, the assumption of Gaussian posterior {\pdf} cannot be easily overcome and for 202 | systems that show out non-Gaussian distributions of the state vector another approach have to be 203 | taken.~\cite{AruMasGor:02} One such approach can be a Monte Carlo-based \emph{particle filter} 204 | presented in the next section. 205 | 206 | \begin{figure}[h] 207 | \centering 208 | \includegraphics[width=\textwidth,keepaspectratio=true]{KF.pdf} 209 | \caption[Example run of the Kalman filter]{Example run of the Kalman filter. Lines are actual 210 | (hidden) state, dots estimation means of various implementations (all yielding the same values).} 211 | \label{fig:KF} 212 | \end{figure} 213 | 214 | \section{Particle Filter} \label{sec:ParticleFilter} 215 | 216 | Particle filters represent an approximate solution of the problem of the recursive Bayesian 217 | estimation, 218 | thus can be considered \emph{suboptimal} methods. The underlying algorithm described below is most 219 | commonly named \emph{sequential importance sampling (SIS)}. The biggest advantage of the particle filtering 220 | is that requirements posed on the modelled system are much weaker than those assumed by optimal methods 221 | such as the Kalman filter. Simple form of the particle filter presented in this section (that assumes that 222 | modelled system has Markov property) requires only the knowledge of {\pdf} \(p(x_t|x_{t-1})\) 223 | representing the process model and the knowledge of \(p(y_t|x_t)\) representing the observation 224 | model.\footnote{both {\pdfs} are generally time-varying and their knowledge for all \(t\) is needed, 225 | but their representation (parametrised by conditioning variable) is frequently constant in time in 226 | practical applications.} 227 | 228 | The sequential importance sampling approximates the posterior density by a weighted empirical 229 | {\pdf} \eqref{eq:PFAPost}. 230 | \begin{align} 231 | p(x_t | y_{1:t}) \approx \sum_{i=1}^N \omega_t^{(i)} \delta(x_t - x_t^{(i)}) \label{eq:PFAPost} \\ 232 | \forall i \in \mathbb{N} \;\; i \leq N: \omega_i \geq 0 \quad\quad \sum_{i=1}^N \omega_i = 1 \notag 233 | \end{align} 234 | 235 | In \eqref{eq:PFAPost} \(x_t^{(i)}\) denotes value of i-th \emph{particle}: possible state of the 236 | system at time \(t\); 237 | \(\omega_t^{(i)}\) signifies weight of i-th particle at time \(t\): scalar value proportional to 238 | expected probability of the system being in state in small neighbourhood of \(x_t^{(i)}\); 239 | \(N\) denotes total number of particles\footnote{\(N\) is assumed to be 240 | arbitrary but fixed positive integer for our uses. Variants of the particle filter exist that use 241 | adaptive number of particles, these are not discussed here.}, a significant tunable parameter 242 | of the filter. 243 | 244 | As the initial step of the described particle filter, \(N\) random particles are sampled 245 | from the initial {\pdf} 246 | \(p(x_0)\). Let \(i \in \mathbb{N} \;\; i \leq N\), transition \(t-1 \; \rightarrow \; t\) can be 247 | performed as follows: 248 | \begin{enumerate} 249 | \item for each \(i\) compute \(x_t^{(i)}\) by random sampling from conditional {\pdf} 250 | \(p(x_t|x_{t-1})\) where \(x_{t-1}^{(i)}\) substitutes \(x_{t-1}\) in condition. This step 251 | can be interpreted as a simulation of possible system state developments. 252 | \item for each \(i\) compute weight \(\omega_t^{(i)}\) using \eqref{eq:PFWeightUpdate} 253 | by taking observation \(y_t\) into account. \(x_t\) is substituted by \(x_t^{(i)}\) in 254 | condition in \eqref{eq:PFWeightUpdate}. Simulated system states are confronted with reality 255 | through observation. 256 | \begin{equation} \label{eq:PFWeightUpdate} 257 | \omega_t^{(i)} = p(y_t | x_t) \omega_{t-1}^{(i)} 258 | \end{equation} 259 | \item normalise weights according to \eqref{eq:PFWeightNormalise} so that approximation of 260 | posterior {\pdf} integrates to one. 261 | \begin{equation} \label{eq:PFWeightNormalise} 262 | \omega_t^{(i)} = \frac{\omega_t^{(i)}}{\sum_{j=1}^N \omega_t^{(j)}} 263 | \end{equation} 264 | \end{enumerate} 265 | 266 | Relative computational ease of described algorithm comes with 267 | cost: first, the particle filter is in principle non-deterministic because of the random sampling in 268 | step~1, in other words, the particle filter is essentially a Monte Carlo method; second, appropriate 269 | number of particles \(N\) has to be chosen --- too small \(N\) can lead to significant approximation 270 | error while inadequately large \(N\) can make the particle filter infeasibly time-consuming. It can be 271 | proved that the particle filter converges to true posterior density as \(N\) approaches 272 | infinity and certain other assumptions hold~\cite{CriDou:02}, therefore the number of particles 273 | should be chosen as a~balance of accuracy and speed. 274 | 275 | Only two operations with {\pdfs} were needed: sampling from \(p(x_t|x_{t-1})\) and evaluating 276 | \(p(y_t | x_t)\) in known point. Sometimes sampling from \(p(x_t|x_{t-1})\) is not 277 | feasible\footnote{but can be replaced by evaluation in known point.} and/or better results are 278 | expected by taking an observation \(y_t\) into account during sampling (step~1). This can be 279 | achieved by introducing so-called \emph{proposal density} (sometimes \emph{importance density}) 280 | \(q(x_t|x_{t-1}, y_t)\). Sampling in step~1 then uses \(q(x_t|x_{t-1}, y_t)\) instead, where \(x_{t-1}\) in 281 | condition is substituted by \(x_{t-1}^{(i)}\). Weight computation in step~2 have to be replaced with 282 | \eqref{eq:PFWeightUpdateProp} that compensates different sampling distribution (every occurrence of 283 | \(x_t\), \(x_{t-1}\) in the mentioned formula has to be substituted by \(x_t^{(i)}\) and \(x_{t-1}^{(i)}\) 284 | respectively). See \cite{AruMasGor:02} for a derivation of these formulas and for a discussion about 285 | choosing adequate proposal density. 286 | \begin{equation} \label{eq:PFWeightUpdateProp} 287 | \omega_t^{(i)} = \frac{p(y_t|x_t)p(x_t|x_{t-1})}{q(x_t|x_{t-1}, y_t)} \omega_{t-1}^{(i)} 288 | \end{equation} 289 | 290 | Particle filters also suffer from a phenomenon known as \emph{sample impoverishment} or 291 | \emph{degeneracy problem}: after a few iterations all but one particles' weight falls close to 292 | zero.\footnote{it has been shown that variance of particle weights continually raises as algorithm 293 | progresses.~\cite{AruMasGor:02}} 294 | One technique to diminish this is based on careful choice of proposal density (as explained in 295 | \cite{AruMasGor:02}), a second one is to add additional \emph{resample} step to the above 296 | algorithm: 297 | \begin{enumerate}[resume] % so that enumeration starts with number 4 298 | \item for each \(i\) resample \(x_t^{(i)}\) from approximate posterior {\pdf} 299 | \(\sum_{i=1}^N \omega_t^{(i)} \delta(x_t - x_t^{(i)})\) and reset all weights to \(\frac{1}{N}\). 300 | Given that sampling is truly random and independent this means that each particle is in 301 | average copied \(n_i\) times, where \(n_i\) is roughly proportional to particle weight: 302 | \(n_i \approx \omega_t^{(i)} N\). Statistics of posterior {\pdf} are therefore (roughly and 303 | on average) maintained while low-weight particles are eliminated. 304 | \end{enumerate} 305 | Step~4 therefore facilitates avoidance of particles with negligible weight by replacing them with more weighted 306 | ones. Such enhanced algorithm is known as \emph{sequential importance resampling (SIR)}. 307 | 308 | Because particle resampling is computationally expensive operation, a technique can be used where 309 | resampling is skipped in some iterations, based on the following idea: 310 | a measurement of degeneracy can be obtained by computing an 311 | approximate of \emph{effective sample size} \(N_{\text{eff}}\) at given time \(t\) using 312 | \eqref{eq:PFNeff}.~\cite{AruMasGor:02} 313 | \begin{equation} \label{eq:PFNeff} 314 | N_{\text{eff}} \approx \left( \sum_{i=1}^N \left( \omega_t^{(i)} \right)^2 \right)^{-1} 315 | \end{equation} 316 | Very small \(N_{\text{eff}}\) compared to \(N\) signifies a substantial loss of ``active'' particles, 317 | which is certainly undesirable as it hurts accuracy while leaving computational demands unchanged. 318 | Step~4 is then performed only when \(N_{\text{eff}}\) falls below certain threshold. 319 | 320 | Recursive Bayesian estimation using SIR methods can be applied to a wide range of dynamic systems 321 | (even to those where more specialised methods fail) and can be tuned with number of particles \(N\) and 322 | proposal density \(q\). On the other hand a method specially designed for a given system easily 323 | outperforms general particle filter in terms of speed and accuracy. 324 | 325 | \section{Marginalized Particle Filter} 326 | 327 | Main sources of this section are \cite{SchGusNor:05} and \cite{SchKarGus:06}. 328 | 329 | The marginalized particle filter (sometimes \emph{Rao-Blackwellized particle filter}) is an 330 | extension to the particle filter that more accurately approximates the optimal Bayesian solution 331 | provided that the {\pdf} representing the process model \(p(x_t|x_{t-1})\) can be obtained in 332 | a special form. Suppose that the state vector can be divided into two parts \(a_t\) and \(b_t\) 333 | \eqref{eq:MPFStateVec} and that the process model {\pdf} can be expressed as a product of two 334 | {\pdfs} \eqref{eq:MPFProcessMod}, where \(p(a_t|a_{t-1} b_t)\) is analytically tractable (in general). 335 | We present a simple variant of the marginalized particle filter where, given \(b_t\), process and 336 | observation model of the \(a_t\) part are linear with normally-distributed noise. The Kalman filter 337 | can be used to estimate \(a_t\) part of the state vector in this case. 338 | \begin{align} 339 | x_t &= (a_t, b_t) \label{eq:MPFStateVec} \\ 340 | p(x_t|x_{t-1}) &= p(a_t, b_t|a_{t-1}, b_{t-1}) = p(a_t|a_{t-1}, b_t) p(b_t | b_{t-1}) \label{eq:MPFProcessMod} 341 | \end{align} 342 | 343 | The posterior {\pdf} \eqref{eq:MPFPosterior} can be represented as a product of a weighted empirical 344 | distribution and a normal distribution. Each (i-th) particle is thus associated with its Kalman filter 345 | (representing \(a_t^{(i)}\) part) and \(b_t^{(i)}\) quantity. 346 | \begin{equation} \label{eq:MPFPosterior} 347 | p(a_t, b_t | y_t) = \sum_{i=1}^N \omega_i p(a_t|y_{1:t},b_{1:t}^{(i)}) \; \delta(b_t - b_t^{(i)}) 348 | \end{equation} 349 | 350 | In \eqref{eq:MPFPosterior} \(N\) denotes the total number of particles, \(\omega_i\) denotes weight 351 | of i-th particle, \(p(a_t|y_{1:t},b_{1:t}^{(i)})\) is posterior {\pdf} of i-th Kalman 352 | filter and \(b_t^{(i)}\) is a value of the \(b_t\) part of i-th particle. 353 | 354 | The algorithm of the described variant of the marginalized particle filter follows, note the 355 | similarities with the ordinary particle filter: at first, generate \(N\) \(b_t\) random samples from 356 | the initial distribution \(p(b_0)\). Then the following procedure can be repeated for each measurement 357 | \(y_t\): 358 | \begin{enumerate} 359 | \item for each \(i\) compute \(b_t^{(i)}\) by random sampling from conditional {\pdf} 360 | \(p(b_t|b_{t-1})\) where \(b_{t-1}^{(i)}\) substitutes \(b_{t-1}\) in condition. 361 | \item for each \(i\) compute the posterior {\pdf} \(p(a_t|y_{1:t},b_{1:t}^{(i)})\) of the i-th 362 | Kalman filter using \(y_t\) and \(b_t^{(i)}\). 363 | \item for each \(i\) update weight \(\omega_i\) using the formula \eqref{eq:MPFWeightUpdate} where 364 | marginal likelihood of the i-th Kalman filter \eqref{eq:KalmanEvidence} is used. 365 | \begin{equation} \label{eq:MPFWeightUpdate} 366 | \omega_i = p(y_t|y_{1:t-1},b_{1:t}^{(i)}) \; \omega_i 367 | \end{equation} 368 | \item normalise weights (as described in the previous section). 369 | \item resample particles (as described in the previous section). 370 | \end{enumerate} 371 | 372 | It has been demonstrated in various publications that the marginalised particle filter outperforms 373 | the ordinary particle filter in both better accuracy and lower computational demands. Where 374 | applicable, it therefore forms an excellent alternative to traditional particle filtering. 375 | -------------------------------------------------------------------------------- /thesis/chap4-conclusion.tex: -------------------------------------------------------------------------------- 1 | \clearpage % so that table of contents mentions correct page 2 | \phantomsection % so that hyperref makes correct reference 3 | \addcontentsline{toc}{chapter}{Conclusion} 4 | \chapter*{Conclusion} 5 | \vspace{-3mm} % HACK aby se conclusion veslo na jednu stranku 6 | The theory of Bayesian filtering is introduced in the first chapter and the \emph{optimal 7 | Bayesian solution} of the problem of recursive estimation is derived. Continues a survey of 8 | well-known Bayesian filtering methods --- the Kalman filtering, particle filtering and the 9 | marginalized particle filtering is described and properties of individual algorithms are discussed. 10 | 11 | The second chapter contains a software analysis performed with the aim to identify the best 12 | approach to software development and programming language for a desired library for Bayesian 13 | filtering. Object-oriented approach is chosen along with the Python programming language, which is 14 | found optimal except its potentially significant computational overhead. Cython is evaluated for the 15 | task to improve Python performance with great success: a simple Python algorithm was 60\x\ faster 16 | when compiled using Cython. 17 | 18 | The last chapters presents the PyBayes library that was developed as a part of this thesis. PyBayes 19 | builds on the software analysis performed in the previous chapter and is therefore object-oriented 20 | and uses Python/Cython combination as its implementation environment and implements all presented 21 | Bayesian filtering methods. To compare performance of Python/Cython combination in a real-world 22 | example, the Kalman filter from PyBayes is benchmarked against MATLAB and C++ implementations from 23 | BDM~\cite{BDM} with favourable results. 24 | 25 | \noindent{}We believe that the \textbf{key contributions} of this thesis are: 26 | \begin{itemize} 27 | \item The performed software analysis, that can be reused for a wide variety of software 28 | projects. In particular, we have shown that the choice of a high-level and convenient 29 | language such as Python is \emph{not necessarily} the enemy of speed. The analysis includes 30 | benchmarks with quite surprising results that show that Cython and PyPy are great speed 31 | boosters of Python. 32 | \item The PyBayes library itself. While it is not yet feature-complete, it provides a solid base 33 | for future development and is unique due to its dual-mode approach: it can be both treated 34 | as ordinary Python package with all the convenience it brings or compiled using Cython for 35 | performance gains. 36 | \end{itemize} 37 | \textbf{Future work} includes extending PyBayes with more filtering algorithms (non-liner Kalman 38 | filter variants etc.) in the long term and fixing little inconveniences that currently exist in 39 | PyBayes in the sort term; version 0.4 that would incorporate all future changes mentioned in the 40 | third chapter is planned to be released within a few months. We are also looking forward to 41 | incorporate emerging projects into our software analysis, for example the PyPy project looks very 42 | promising. 43 | -------------------------------------------------------------------------------- /thesis/coverage.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/coverage.pdf -------------------------------------------------------------------------------- /thesis/cvut-logo-bw-600.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/cvut-logo-bw-600.pdf -------------------------------------------------------------------------------- /thesis/diagrams/PyBayes.dia: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/PyBayes.dia -------------------------------------------------------------------------------- /thesis/diagrams/PyBayes.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/PyBayes.pdf -------------------------------------------------------------------------------- /thesis/diagrams/emp_pdfs.dia: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/emp_pdfs.dia -------------------------------------------------------------------------------- /thesis/diagrams/emp_pdfs.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/emp_pdfs.pdf -------------------------------------------------------------------------------- /thesis/diagrams/filters.dia: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/filters.dia -------------------------------------------------------------------------------- /thesis/diagrams/filters.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/filters.pdf -------------------------------------------------------------------------------- /thesis/diagrams/gaussian_pdfs.dia: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/gaussian_pdfs.dia -------------------------------------------------------------------------------- /thesis/diagrams/gaussian_pdfs.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/gaussian_pdfs.pdf -------------------------------------------------------------------------------- /thesis/diagrams/page_setup.dia: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/page_setup.dia -------------------------------------------------------------------------------- /thesis/diagrams/particile_filters.dia: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/particile_filters.dia -------------------------------------------------------------------------------- /thesis/diagrams/pdfs.dia: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/pdfs.dia -------------------------------------------------------------------------------- /thesis/diagrams/pdfs.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/pdfs.pdf -------------------------------------------------------------------------------- /thesis/diagrams/pdfs_model_outdated.dia: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/pdfs_model_outdated.dia -------------------------------------------------------------------------------- /thesis/diagrams/rvs.dia: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/rvs.dia -------------------------------------------------------------------------------- /thesis/diagrams/rvs.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/diagrams/rvs.pdf -------------------------------------------------------------------------------- /thesis/prezentace/.gitignore: -------------------------------------------------------------------------------- 1 | prezentace.pdf 2 | -------------------------------------------------------------------------------- /thesis/prezentace/air-header.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/prezentace/air-header.png -------------------------------------------------------------------------------- /thesis/prezentace/beamerthemeAir.sty: -------------------------------------------------------------------------------- 1 | \usetheme{Rochester} 2 | 3 | \pgfdeclareimage[width=1.0\paperwidth]{air-header}{air-header} 4 | 5 | \setbeamertemplate{blocks}[rounded][shadow=true] 6 | \setbeamercovered{transparent} 7 | 8 | \beamer@headheight=0.13\paperwidth 9 | 10 | \definecolor{airorange}{rgb}{.953 ,.502 ,.04} 11 | \definecolor{airdarkblue}{HTML}{064F6F} 12 | \definecolor{airlightblue}{HTML}{0099d5} 13 | \definecolor{airgray}{HTML}{686868} 14 | \definecolor{airlightgray}{HTML}{eeeeee} 15 | \definecolor{airblue}{rgb}{.13 ,.43 ,.68} 16 | \setbeamercolor*{Title bar}{fg=white} % fg=airdarkblue 17 | \setbeamercolor*{Location bar}{fg=airorange,bg=airlightgray} 18 | \setbeamercolor*{frametitle}{parent=Title bar} 19 | \setbeamercolor*{block title}{bg=airlightblue,fg=white} 20 | \setbeamercolor*{block body}{bg=airlightgray,fg=airgray} 21 | \setbeamercolor*{normal text}{bg=white,fg=airgray} 22 | \setbeamercolor*{section in head/foot}{bg=airblue,fg=black} 23 | 24 | \usecolortheme[named=airorange]{structure} 25 | 26 | \setbeamerfont{section in head/foot}{size=\tiny,series=\normalfont} 27 | \setbeamerfont{frametitle}{size=\Large} 28 | 29 | %\setbeamertemplate{headline} 30 | \setbeamertemplate{frametitle} 31 | { 32 | \vskip-0.25\beamer@headheight 33 | \vskip-\baselineskip 34 | \vskip-0.2cm 35 | \hskip0.7cm\usebeamerfont*{frametitle}\insertframetitle 36 | \vskip-0.10em 37 | \hskip0.7cm\usebeamerfont*{framesubtitle}\insertframesubtitle 38 | } 39 | 40 | \setbeamertemplate{headline} 41 | { 42 | \pgfuseimage{air-header} 43 | \vskip -1.95cm 44 | \linethickness{0.0pt} 45 | 46 | \framelatex{ 47 | \begin{beamercolorbox}[wd=\paperwidth,ht=0.3\beamer@headheight]{Title bar} 48 | \usebeamerfont{section in head/foot}% 49 | \insertsectionnavigationhorizontal{0pt}{\hskip0.22cm}{}% 50 | \end{beamercolorbox}} 51 | 52 | \framelatex{ 53 | \begin{beamercolorbox}[wd=\paperwidth,ht=0.7\beamer@headheight]{Title bar} 54 | \end{beamercolorbox}} 55 | } 56 | 57 | \setbeamertemplate{footline} 58 | { 59 | \linethickness{0.25pt} 60 | \framelatex{ 61 | \begin{beamercolorbox}[leftskip=.3cm,wd=\paperwidth,ht=0.3\beamer@headheight,sep=0.1cm]{Location bar} 62 | \usebeamerfont{section in head/foot}% 63 | \insertshortauthor~|~\insertshorttitle 64 | \hfill 65 | \insertframenumber/\inserttotalframenumber 66 | \end{beamercolorbox}} 67 | } 68 | 69 | -------------------------------------------------------------------------------- /thesis/prezentace/edgy-results.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/strohel/PyBayes/76d3ad12c20e29c5aadb688c4323fb0f1784f8ec/thesis/prezentace/edgy-results.pdf -------------------------------------------------------------------------------- /thesis/prezentace/prezentace.tex: -------------------------------------------------------------------------------- 1 | \documentclass[12pt]{beamer} 2 | 3 | \usetheme{Air} 4 | \usepackage[czech]{babel} 5 | \usepackage[utf8x]{inputenc} 6 | 7 | \title{Prostředí pro implementaci algoritmů Bayesovské filtrace} 8 | \author[Matěj Laitl]{Matěj Laitl\\ vedoucí práce: Ing. Václav Šmídl, Ph.D.; ÚTIA} 9 | \date{\today} 10 | 11 | \begin{document} 12 | 13 | \frame{\titlepage} 14 | 15 | \section*{} 16 | \begin{frame} 17 | \frametitle{Obsah} 18 | \tableofcontents[section=1,hidesubsections] 19 | \end{frame} 20 | 21 | \AtBeginSection[] 22 | { 23 | \frame 24 | { 25 | \frametitle{Přehled} 26 | \tableofcontents[currentsection,hideallsubsections] 27 | } 28 | } 29 | 30 | \AtBeginSubsection[] 31 | { 32 | \frame 33 | { 34 | \frametitle{Přehled} 35 | \tableofcontents[sectionstyle=show/hide,subsectionstyle=show/shaded/hide] 36 | } 37 | } 38 | 39 | 40 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 41 | %%%%%%%%%% Content starts here %%%%%%%%%% 42 | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 43 | 44 | \section*{Zadání} 45 | 46 | \begin{frame} 47 | \frametitle{Zadání} 48 | 49 | \begin{block}{Úkoly práce} 50 | \begin{itemize} 51 | \item seznámit se s metodou rekurzivní Bayesovské identifikace (Bayesovské filtrace) 52 | \item softwarová analýza - vhodnost přístupů a programových prostředí 53 | \item implementace několika metod Bayesovské filtrace 54 | \item zveřejnění knihovny jako open-source projektu 55 | \end{itemize} 56 | \end{block} 57 | \end{frame} 58 | 59 | \section{Rekurzivní Bayesovská identifikace} 60 | 61 | \begin{frame} 62 | \frametitle{Formulace problému 1/2} 63 | \framesubtitle{Dynamický systém} 64 | 65 | \begin{block}{model procesu} 66 | \(x_t = f(x_{t-1}, v_{t-1})\) \hspace{3cm} \(x_t\) .. stavový vektor 67 | 68 | \begin{itemize} 69 | \item \(v_t\) náhodný šum, iid 70 | \item → definuje pdf\footnote{hustota pravděpodobnosti} \(p(x_t|x_{t-1})\) 71 | \end{itemize} 72 | \end{block} 73 | 74 | \begin{block}{model pozorování} 75 | \(y_t = h(x_t, w_t)\) \hspace{3cm} \(y_t\) .. vektor pozorování 76 | 77 | \begin{itemize} 78 | \item \(w_t\) náhodný šum, iid 79 | \item → definuje pdf \(p(y_t|x_t)\) 80 | \end{itemize} 81 | \end{block} 82 | \end{frame} 83 | 84 | \begin{frame} 85 | \frametitle{Formulace problému 2/2} 86 | \framesubtitle{Cíle} 87 | 88 | {\large Úkol: 89 | \begin{itemize} 90 | \item odhadnout \(x_t\) z posloupnosti meření \(y_{1:t}\) 91 | \item precizněji: spočítat pdf \(p(x_t | y_{1:t})\) 92 | \end{itemize} 93 | } 94 | 95 | \begin{block}{Máme} 96 | \begin{itemize} 97 | \item \(p(x_t|x_{t-1})\) \hspace{1cm} speciálně \(p(x_0)\) 98 | \item \(p(y_t|x_t)\) 99 | \end{itemize} 100 | \end{block} 101 | 102 | \begin{example} 103 | \begin{itemize} 104 | \item robotika: lokalizace, sledování; avionika 105 | \item environmentální simulace: síření radioaktivního spadu 106 | \end{itemize} 107 | \end{example} 108 | \end{frame} 109 | 110 | % \begin{frame} 111 | % \frametitle{Matematický aparát} 112 | % \begin{block}{Řetězové pravidlo pro pdf} 113 | % \[P(AB|C) = P(A|BC) P(B|C)\] 114 | % \end{block} 115 | % 116 | % \begin{block}{Bayesova věta} 117 | % \[P(A|BC) = \frac{P(B|AC)P(A|C)}{P(B|C)}\] 118 | % \end{block} 119 | % 120 | % \begin{block}{Marginalizace pdf} 121 | % \[p(x, y) = \int_{-\infty}^{\infty} p(x, y, z) \; \mathrm{d} z\] 122 | % \end{block} 123 | % \end{frame} 124 | 125 | \begin{frame} 126 | \frametitle{Teoretické řešení} 127 | \framesubtitle{Rekurzivního algoritmus (zjednodušeno; Markovova vlastnost)} 128 | 129 | {\small Známe \(p(x_{t-1} | y_{1:t-1})\), odvoďme apriorní pdf \eqref{eq:PriorPdf}.} 130 | {\scriptsize (marg., řetěz. prav.)} 131 | \begin{align} 132 | p(x_t | y_{1:t-1}) &= \int_{-\infty}^{\infty} p(x_t, x_{t-1} | y_{1:t-1}) 133 | \; \mathrm{d} x_{t-1} \notag \\ 134 | % 135 | p(x_t | y_{1:t-1}) &= \int_{-\infty}^{\infty} p(x_t | x_{t-1}) p(x_{t-1} | y_{1:t-1}) 136 | \; \mathrm{d} x_{t-1} \label{eq:PriorPdf} 137 | \end{align} 138 | 139 | {\small Přišlo pozorování \(y_t\), zakomponujme ho.} 140 | {\scriptsize (Bayesova věta)} 141 | \begin{align} 142 | p(x_t | y_{1:t}) &= \frac{p(y_t | x_t) p(x_t | y_{1:t-1})}{p(y_t | y_{1:t-1})} 143 | \label{eq:PosteriorPdf} 144 | \end{align} 145 | 146 | {\small Odvodili jsme aposteriorní pdf \eqref{eq:PosteriorPdf} pro \(x_t\).} 147 | % Čitatel známý, jmenovatel je konstanta - buď netřeba nebo lze dopočítat marginalizací/z podmínky 148 | % normality. 149 | \end{frame} 150 | 151 | \begin{frame} 152 | \frametitle{Problémy analytického řešení} 153 | \framesubtitle{Praktické metody Bayesovské filtrace} 154 | 155 | \begin{alertblock}{Problém!} 156 | Odvozené analytické řešení je velmi obtížné a nepraktické počítat pro více-dimenzionální 157 | systémy 158 | \end{alertblock} 159 | 160 | \begin{block}{Zjednodušení problému} 161 | \begin{itemize} 162 | \item Kalmanův filtr: Gaussovské šumy, lineární systém → zjednodušení 163 | na algebraické operace s kovariančními maticemi 164 | \end{itemize} 165 | \end{block} 166 | 167 | \begin{block}{Aproximace} 168 | \begin{itemize} 169 | \item Particle filtr: odhaduje aposteriorní pdf empirickou pdf, 170 | nedeterministický (Monte Carlo - samplování z pdf) 171 | \item Marginalizovaný particle filtr (Kalmanův + particle) 172 | \end{itemize} 173 | \end{block} 174 | 175 | \end{frame} 176 | 177 | \section{Výsledky softwarové analýzy} 178 | 179 | \begin{frame} 180 | \frametitle{Východiska} 181 | 182 | {\large Cíl: vyvinout konkurenceschopnou knihovnu pro Bayesovské filtrování a jeho aplikace} 183 | 184 | \vspace{8mm} 185 | 186 | \begin{block}{Nejdůležitější požadavky} 187 | \begin{itemize} 188 | \item snadné a rychlé použití {\scriptsize (→ vysokoúrovňový jazyk)} 189 | \item vysoký výpočetní výkon {\scriptsize (v rozporu s předchozím)} 190 | \item rozšiřitelnost, přenositelnost, nezávislost na komerčních platformách 191 | \end{itemize} 192 | \end{block} 193 | \end{frame} 194 | 195 | \begin{frame} 196 | \frametitle{Paradigmata, Programovací jazyky} 197 | 198 | \begin{block}{Vhodnost implementačních metodik} 199 | Objektově orientované programování nejlépe vystihuje matematickou strukturu teorie 200 | \end{block} 201 | 202 | \begin{block}{Implementační prostředí 1} 203 | \begin{itemize} 204 | \item C++ 205 | \begin{itemize} 206 | \item + rozšířenost, rychlost, existující knihovny pro Bayes.. 207 | \item - nízká produktivita, vysoké vstupní bariéry → nevhodné 208 | \end{itemize} 209 | \item Matlab 210 | \begin{itemize} 211 | \item + rozšířenost, vysoká programátorská produktivita 212 | \item - copy-on-write, nevhodné pro OOP, proprietární → nevhodný 213 | \end{itemize} 214 | \end{itemize} 215 | \end{block} 216 | \end{frame} 217 | 218 | \begin{frame} 219 | \frametitle{Programovací jazyky redux} 220 | 221 | \begin{block}{Implementační prostředí 2} 222 | \begin{itemize} 223 | \item Python 224 | \begin{itemize} 225 | \item + vysoká produktivita, existující mat. knihovny (NumPy), rozšířenost, komunita 226 | \item - vysoká režie (interpretovaný, dynamický jazyk) → sám o sobě nevhodný 227 | \end{itemize} 228 | \item Cython 229 | \begin{itemize} 230 | \item překládá Python kód do C 231 | \item \textbf{postupná} optimalizace stávajícího Python kódu 232 | \item naměřeno: 60x zrychlení pro numerickou integraci\footnote{dokazuje zejména 233 | pomalost Pythonu} 234 | \item naměřeno: dodatečné 13x zrychlení při paralelizaci na 16-jádrovém CPU (OpenMP) 235 | \end{itemize} 236 | \end{itemize} 237 | \end{block} 238 | \end{frame} 239 | 240 | \begin{frame} 241 | \frametitle{Výkonnostní porovnání} 242 | \framesubtitle{C, Python, Cython, PyPy} 243 | 244 | Relativní rychlost numerické integrace 245 | \vspace{5mm} 246 | 247 | \includegraphics[width=\textwidth,keepaspectratio=true,clip=true,trim=30mm 166mm 30mm 248 | 54mm]{./edgy-results.pdf} 249 | \end{frame} 250 | 251 | \section{PyBayes} 252 | 253 | \begin{frame} 254 | \frametitle{PyBayes} 255 | 256 | \begin{block}{PyBayes knihovna} 257 | \begin{itemize} 258 | \item open-source, otevřený vývoj na \url{http://github.com/strohel/PyBayes} 259 | \item možno interpretovat Pythonem / kompilovat Cythonem (optimalizace) 260 | \item Cython verze je 2 - 5x rychlejší 261 | \end{itemize} 262 | \end{block} 263 | 264 | \begin{block}{Implementováno v PyBayes} 265 | \begin{itemize} 266 | \item základní i podmíněné pdf, produkt pdf, řetězové pravidlo 267 | \item Kalmanův filtr, particle filtr, marginalizovaný PF 268 | \item kompletní dokumentace, unit-testy 269 | \end{itemize} 270 | \end{block} 271 | \end{frame} 272 | 273 | \begin{frame} 274 | \frametitle{Výkonnostní srovnání PyBayes} 275 | \framesubtitle{Čas na 3000 iterací Kalmanova filtru} 276 | 277 | \includegraphics[width=\textwidth,keepaspectratio=true,clip=true,trim=19mm 85mm 20mm 278 | 111mm]{../KFPerf.pdf} 279 | \end{frame} 280 | 281 | 282 | \begin{frame} 283 | \begin{block}{Budoucnost} 284 | \begin{itemize} 285 | \item další filtrační metody: nelineární Kalmanovy filtry... 286 | \item další implementační prostředí: PyPy... 287 | \end{itemize} 288 | \end{block} 289 | 290 | \vspace{1cm} 291 | {\huge Děkuji za pozornost} 292 | 293 | \vspace{1cm} 294 | \begin{flushright} 295 | Matěj Laitl 296 | 297 | \structure{\footnotesize{matej@laitl.cz}} 298 | \end{flushright} 299 | \end{frame} 300 | 301 | \end{document} 302 | -------------------------------------------------------------------------------- /thesis/startpages.tex: -------------------------------------------------------------------------------- 1 | \newcommand{\cvut}{České vysoké učení technické v~Praze} 2 | \newcommand{\fjfi}{Fakulta jaderná a fyzikálně inženýrská} 3 | \newcommand{\km}{Katedra matematiky} 4 | \newcommand{\obor}{Inženýrská Informatika} 5 | \newcommand{\zamereni}{Softwarové inženýrství a matematická informatika} 6 | 7 | \newcommand{\nazevcz}{Prostředí pro implementaci algoritmů Bayesovské filtrace} 8 | \newcommand{\nazeven}{Implementation environment for Bayesian filtering algorithms} 9 | \newcommand{\autor}{Matěj Laitl} 10 | \newcommand{\rok}{2011} 11 | \newcommand{\vedouci}{Ing. Václav Šmídl, Ph.D.} 12 | 13 | \newcommand{\pracovisteVed}{Oddělení adaptivních systémů \\ 14 | Ústav teorie informace a automatizace \\ 15 | Akademie věd České republiky} 16 | \newcommand{\konzultant}{---} 17 | \newcommand{\pracovisteKonz}{} 18 | 19 | \newcommand{\klicova}{Bayesovská filtrace, softwarová analýza, Python, Cython} % TODO: dalsi? 20 | \newcommand{\keyword}{Bayesian filtering, software analysis, Python, Cython} 21 | % Abstrakt práce: (cca 7 vět, min. 80 slov) 22 | \newcommand{\abstrCZ}{Bayesovská filtrace je velmi použitelný přístup k odhadování dynamických 23 | systémů, který má mnoho aplikací v robotice, environmentálních simulacích a dalších oborech. Tato 24 | práce předkládá popis Kalmanova filtru, částicového filtru a marginalizovaného částicového filtru, 25 | který využívá myšlenky obou předchozích. Poté je provedena softwarová analýza, která má za úkol 26 | určit nejvhodnější implementační prostředí pro softwarovou knihovnu poskytující metody Bayesovské 27 | filtrace. Nejprve jsou posouzeny obecné přístupy k vývoji software, z nichž je jako nejpoužitelnější 28 | zvolen objektově orientovaný, následuje porovnání programovacích jazyků C++, MATLAB a Python. 29 | Python zvítězí a v je něm napsána knihovna pro Bayesovskou filtraci nazvaná PyBayes; nakonec je 30 | dosaženo výrazného zrychlení Pythonového kódu použitím Cythonu.} 31 | \newcommand{\abstrEN}{Bayesian filtering is a vital approach to dynamic system estimation that has 32 | wide applications in robotics, environmental simulations and much more. This text briefly introduces 33 | the Kalman filter, the particle filter and the marginalized particle filter which combines ideas of 34 | both. Software analysis is performed with the aim to identify the most suitable implementation 35 | environment for a software library for Bayesian filtering. Various paradigms of software development 36 | and are discussed among which the object-oriented programming is chosen; C++, MATLAB and Python 37 | languages are evaluated. Python is determined most appropriate and a Python software library named 38 | PyBayes is developed; dramatic performance gains are measured when Cython is used to speed up 39 | Python.} 40 | 41 | %%% zde zacina kresleni dokumentu 42 | 43 | % titulní strana 44 | \thispagestyle{empty} 45 | 46 | \begin{center} 47 | {\Large \bf \cvut\\[2mm] \fjfi } 48 | \vspace{10mm} 49 | 50 | \begin{tabular}{c} 51 | {\bf \km}\\ 52 | {\bf Obor: \obor}\\ 53 | {\bf Zaměření: \zamereni} 54 | \end{tabular} 55 | 56 | \vspace{10mm} \epsfysize=20mm \epsffile{cvut-logo-bw-600} \vspace{15mm} 57 | 58 | {\LARGE 59 | \textbf{\nazevcz} 60 | \par} 61 | 62 | \vspace{5mm} 63 | 64 | {\LARGE 65 | \textbf{\nazeven} 66 | \par} 67 | 68 | \vspace{30mm} 69 | {\Large BAKALÁŘSKÁ PRÁCE} 70 | 71 | \end{center} 72 | 73 | \vfill 74 | {\large 75 | \begin{tabular}{rl} 76 | Vypracoval: & \autor\\ 77 | Vedoucí práce: & \vedouci\\ 78 | Rok: & \rok 79 | \end{tabular} 80 | } 81 | 82 | % zadání bakalářské práce 83 | \newpage 84 | \thispagestyle{empty} Před svázáním místo téhle stránky \fbox{vložíte zadání práce} s podpisem 85 | děkana (bude to jediný oboustranný list ve Vaší práci) !!!! 86 | 87 | % prohlášení 88 | \newpage 89 | \thispagestyle{empty} 90 | ~ 91 | \vfill 92 | 93 | 94 | {\noindent}{\LARGE Prohlášení} 95 | 96 | \vspace{0.5cm} 97 | Prohlašuji, že jsem předloženou práci vypracoval samostatně a že jsem uvedl veškerou použitou 98 | literaturu. 99 | 100 | \vspace{5mm}V Praze dne ....................\hfill 101 | \begin{tabular}{c} 102 | ........................................\\ 103 | \autor 104 | \end{tabular} 105 | 106 | % poděkování 107 | \newpage 108 | \thispagestyle{empty} 109 | ~ 110 | \vfill 111 | 112 | {\noindent}{\LARGE Poděkování} 113 | 114 | \vspace{5mm} 115 | Děkuji Ing.\ Václavu Šmídlovi, Ph.D. za vedení mé bakalářské práce, prozíravě položené 116 | softwarové otázky a za poukázání na softwarové projekty (Cython, Sphinx), které se ukázaly 117 | být klíčové pro moji bakalářskou práci a softwarový projekt. 118 | 119 | \begin{flushright} 120 | \autor 121 | \end{flushright} 122 | 123 | % strana s abstraktem 124 | \newpage 125 | \thispagestyle{empty} 126 | 127 | \newbox\odstavecbox 128 | \newlength\vyskaodstavce 129 | \newcommand\odstavec[2]{% 130 | \setbox\odstavecbox=\hbox{% 131 | \parbox[t]{#1}{#2\vrule width 0pt depth 4pt}}% 132 | \global\vyskaodstavce=\dp\odstavecbox 133 | \box\odstavecbox} 134 | \newcommand{\delka}{120mm} 135 | 136 | \noindent\begin{tabular}{@{}ll} 137 | {\em Název práce:} & ~ \\ 138 | \multicolumn{2}{@{}l}{\odstavec{\textwidth}{\bf \nazevcz}} \\[5mm] 139 | {\em Autor:} & \autor \\[5mm] 140 | {\em Obor:} & \obor \\ 141 | {\em Druh práce:} & Bakalářská práce \\[5mm] 142 | {\em Vedoucí práce:} & \odstavec{\delka}{\vedouci \\ \pracovisteVed} \\[5mm] 143 | {\em Konzultant:} & \odstavec{\delka}{\konzultant \\ \pracovisteKonz} \\[5mm] 144 | \multicolumn{2}{@{}l}{\odstavec{\textwidth}{{\em Abstrakt:} ~ \abstrCZ \\ }} \\[5mm] 145 | {\em Klíčová slova:} & \odstavec{\delka}{\klicova} \\[10mm] 146 | 147 | {\em Title:} & ~\\ 148 | \multicolumn{2}{@{}l}{\odstavec{\textwidth}{\bf \nazeven}}\\[5mm] 149 | {\em Author:} & \autor \\[5mm] 150 | \multicolumn{2}{@{}l}{\odstavec{\textwidth}{{\em Abstract:} ~ \abstrEN \\ }} \\[5mm] 151 | {\em Key words:} & \odstavec{\delka}{\keyword} 152 | \end{tabular} 153 | -------------------------------------------------------------------------------- /thesis/thesis.tex: -------------------------------------------------------------------------------- 1 | \documentclass[a4paper,12pt,oneside]{report} 2 | 3 | \newif\ifprint % true = printed version = no colours; false = colours 4 | \printfalse 5 | \newif\ifattachements % true = include fancy start pages and appendices 6 | \attachementsfalse 7 | 8 | \usepackage[english]{babel} 9 | \usepackage[utf8]{inputenc} 10 | \usepackage[T1]{fontenc} 11 | \usepackage{amsmath} 12 | \usepackage{amssymb} 13 | \usepackage{epsfig} 14 | \usepackage{graphicx} 15 | \usepackage{enumitem} % so that spacing for lists can be easily set, resume option etc 16 | \ifattachements 17 | \usepackage{pdfpages} 18 | \fi 19 | \usepackage{fancyvrb} % for Verbatim environment than can be instructed not to page-break 20 | \usepackage[hyphens]{url} % the package is not needed, it needes just to set the parameter 21 | \usepackage[pagebackref=true]{hyperref} % tento balicek by mel byt na konci baliku! 22 | 23 | \hypersetup{ 24 | pdfauthor={Matej Laitl}, 25 | pdftitle={Implementation environment for Bayesian filtering algorithms} 26 | } 27 | 28 | %% Nastavení okraju 29 | \usepackage{calc} 30 | \setlength{\textheight}{\paperheight -3cm -3cm} % horni a dolni okraje 3cm, do tech se musi vejit header a footer 31 | \setlength{\textwidth}{\paperwidth -3.5cm -2.5cm} % levy okraj 3,5cm (na vazbu), pravy okraj 2,5cm 32 | \setlength{\oddsidemargin}{3.5cm -1in} 33 | \setlength{\topmargin}{(\paperheight-\textheight-\headheight-\headsep-\footskip)/2 - 1in} 34 | 35 | %\parindent=0pt % odsazení 1. řádku odstavce 36 | \parskip=6pt % mezera mezi odstavci 37 | 38 | % set that list should not have vertical space 39 | \setlist{nolistsep} 40 | \setlist[1]{leftmargin=\parindent} % set indentation to level 1 lists to be same as paragraph indent 41 | 42 | \ifprint 43 | \hypersetup{pdfborder={0 0 0}} % no borders around links 44 | \else 45 | \hypersetup{colorlinks=true} % colour links instead of borders 46 | \fi 47 | 48 | \pdfminorversion=5 % aby PdfTeX produkovat pdfka verze 1.5 a zejmena nekricel pri jejich vkladani 49 | 50 | % definitions of our own commands: 51 | \newcommand{\pdf}{probability density function} 52 | \newcommand{\pdfs}{probability density functions} 53 | \newcommand{\supp}{\operatorname{supp}} 54 | \newcommand{\dx}{\mathrm{d}x} 55 | \newcommand{\x}{\(\times\)} 56 | 57 | \begin{document} 58 | \VerbatimFootnotes % this must be AFTER preamble 59 | 60 | \ifattachements 61 | \pagenumbering{alph} % just that LaTex does not complain there are 2 pages no. 2. not shown anywhere 62 | \input{startpages.tex} % include some fancy start pages 63 | \fi 64 | 65 | % obsah 66 | \clearpage 67 | \pagenumbering{roman} 68 | \tableofcontents 69 | 70 | \input{chap0-notation.tex} 71 | 72 | \input{chap0-introduction.tex} 73 | 74 | \input{chap1-theory.tex} 75 | 76 | \input{chap2-software.tex} 77 | 78 | \input{chap3-pybayes.tex} 79 | 80 | \input{chap4-conclusion.tex} 81 | 82 | \clearpage % so that table of contents mentions correct page 83 | \phantomsection % so that hyperref makes correct reference 84 | \addcontentsline{toc}{chapter}{\listfigurename} 85 | \listoffigures 86 | 87 | \clearpage % so that table of contents mentions correct page 88 | \phantomsection % so that hyperref makes correct reference 89 | \addcontentsline{toc}{chapter}{\bibname} 90 | \bibliographystyle{plain} 91 | \bibliography{bibliography} 92 | 93 | \ifattachements 94 | % přílohy 95 | \appendix % aby LaTeX cisloval jinak 96 | 97 | \chapter{Contents of the Enclosed CD-ROM} 98 | 99 | \begin{description} 100 | \item[PyBayes-0.3/] \hfill \\ 101 | source code of the PyBayes library, version 0.3. 102 | \item[PyBayes-0.3-API-Docs/] \hfill \\ 103 | PyBayes API Documentation based on version 0.3, HTML version; point your browser to the 104 | \nolinkurl{PyBayes-0.3-API-Docs/index.html} file to view it. 105 | \item[abstract.pdf] \hfill \\ 106 | Czech and English abstract of this text. 107 | \item[PyBayes-0.3-API-Docs.pdf] \hfill \\ 108 | PyBayes API Documentation based on version 0.3, PDF version, hyper-cross\-referenced. 109 | \item[PyBayes-0.3.tar.cz] \hfill \\ 110 | source code of the PyBayes library, version 0.3, packed using tar + gzip for easy 111 | distribution. 112 | \item[thesis.pdf] \hfill \\ 113 | electronic version of this text, hyper-crossreferenced. 114 | \end{description} 115 | 116 | \fi 117 | 118 | \end{document} 119 | -------------------------------------------------------------------------------- /thesis/zadani1.lyx: -------------------------------------------------------------------------------- 1 | #LyX 1.6.7 created this file. For more info see http://www.lyx.org/ 2 | \lyxformat 345 3 | \begin_document 4 | \begin_header 5 | \textclass article 6 | \use_default_options true 7 | \language english 8 | \inputencoding auto 9 | \font_roman default 10 | \font_sans default 11 | \font_typewriter default 12 | \font_default_family default 13 | \font_sc false 14 | \font_osf false 15 | \font_sf_scale 100 16 | \font_tt_scale 100 17 | 18 | \graphics default 19 | \paperfontsize default 20 | \use_hyperref false 21 | \papersize default 22 | \use_geometry false 23 | \use_amsmath 1 24 | \use_esint 1 25 | \cite_engine basic 26 | \use_bibtopic false 27 | \paperorientation portrait 28 | \secnumdepth 3 29 | \tocdepth 3 30 | \paragraph_separation indent 31 | \defskip medskip 32 | \quotes_language english 33 | \papercolumns 1 34 | \papersides 1 35 | \paperpagestyle default 36 | \tracking_changes false 37 | \output_changes false 38 | \author "" 39 | \author "" 40 | \end_header 41 | 42 | \begin_body 43 | 44 | \begin_layout Standard 45 | Mejme system 46 | \begin_inset Formula \begin{eqnarray*} 47 | p(x_{t}|x_{t})\\ 48 | p(y_{t}|x_{t})\end{eqnarray*} 49 | 50 | \end_inset 51 | 52 | jedna z odhadovacich metod, ktera se pouziva je particle filter. 53 | \end_layout 54 | 55 | \begin_layout Subsection* 56 | Particle filter 57 | \end_layout 58 | 59 | \begin_layout Standard 60 | Ten pracuje tak, ze jeho aposteriorno je ve tvaru empiricke hustoty 61 | \begin_inset Formula \begin{equation} 62 | p(x_{t}|y_{1:t})=\sum_{i=1}^{n}w_{i}\delta(x_{t}-x_{t}^{(i)})\label{eq:emp}\end{equation} 63 | 64 | \end_inset 65 | 66 | tedy v podstate uchovava dve pole 67 | \begin_inset Formula $n$ 68 | \end_inset 69 | 70 | cisel 71 | \begin_inset Formula $w=[w_{1}\ldots w_{n}]$ 72 | \end_inset 73 | 74 | , 75 | \begin_inset Formula $x_{t}^{(1:n)}=[x_{t}^{(1)}\ldots x_{t}^{(n)}]$ 76 | \end_inset 77 | 78 | . 79 | \end_layout 80 | 81 | \begin_layout Standard 82 | Momenty distribuce ( 83 | \begin_inset CommandInset ref 84 | LatexCommand ref 85 | reference "eq:emp" 86 | 87 | \end_inset 88 | 89 | ) jsou 90 | \begin_inset Formula \[ 91 | \hat{x}_{t}=\sum_{i=1}^{n}w_{i}x_{t}^{(i)},\,\,\, var(x_{t})=\sum_{i=1}^{n}w_{i}\left(x_{t}^{(i)}-\hat{x}_{t}\right)^{2}.\] 92 | 93 | \end_inset 94 | 95 | Metoda bayes funguje nasledovne: 96 | \end_layout 97 | 98 | \begin_layout Enumerate 99 | nageneruj pocatecni 100 | \begin_inset Formula $x_{0}^{(1:n)}$ 101 | \end_inset 102 | 103 | , a nastav 104 | \begin_inset Formula $w_{i}=1/n,\forall i$ 105 | \end_inset 106 | 107 | 108 | \end_layout 109 | 110 | \begin_layout Enumerate 111 | pro kazde nove mereni 112 | \begin_inset Formula $y_{t}$ 113 | \end_inset 114 | 115 | 116 | \end_layout 117 | 118 | \begin_deeper 119 | \begin_layout Enumerate 120 | vygeneruj nove 121 | \begin_inset Formula $x_{t}$ 122 | \end_inset 123 | 124 | =sample from 125 | \begin_inset Formula $p(x_{t}|x_{t-1})$ 126 | \end_inset 127 | 128 | 129 | \end_layout 130 | 131 | \begin_layout Enumerate 132 | prepocti 133 | \begin_inset Formula $w_{i}=p(y_{t}|x_{t}^{(i)})w_{i},\forall i$ 134 | \end_inset 135 | 136 | 137 | \end_layout 138 | 139 | \begin_layout Enumerate 140 | normalizuj 141 | \begin_inset Formula $w=\frac{w}{\sum w}$ 142 | \end_inset 143 | 144 | 145 | \end_layout 146 | 147 | \begin_layout Enumerate 148 | resample 149 | \begin_inset Formula $w$ 150 | \end_inset 151 | 152 | , 153 | \begin_inset Formula $x_{t}$ 154 | \end_inset 155 | 156 | (viz. 157 | BDM::eEmp::resample) 158 | \end_layout 159 | 160 | \end_deeper 161 | \begin_layout Standard 162 | To je cele. 163 | Aposteriorno je typu empiricka hustota, bayes je dan, vic neni potreba. 164 | \end_layout 165 | 166 | \begin_layout Standard 167 | Snadne pouziti: 168 | \end_layout 169 | 170 | \begin_layout Itemize 171 | zadame prechodove hustoty 172 | \begin_inset Formula $p(\cdot|\cdot)$ 173 | \end_inset 174 | 175 | ve forme CPdf. 176 | \end_layout 177 | 178 | \begin_layout Itemize 179 | zadame 180 | \begin_inset Formula $n$ 181 | \end_inset 182 | 183 | a Pdf ze ktere se generuji prvni vzorky. 184 | \end_layout 185 | 186 | \begin_layout Standard 187 | Vse uz samo bezi. 188 | \end_layout 189 | 190 | \begin_layout Standard 191 | Komplikovanejsi pouziti: 192 | \end_layout 193 | 194 | \begin_layout Standard 195 | Mame system 196 | \begin_inset Formula $x_{t}=[a_{t},b_{t}]$ 197 | \end_inset 198 | 199 | 200 | \begin_inset Formula \begin{eqnarray} 201 | p(x_{t}|x_{t-1}) & = & p(a_{t}|a_{t-1},b_{t})p(b_{t}|b_{t-1})\nonumber \\ 202 | p(y_{t}|x_{t}) & = & p(y_{t}|b_{t})=N(a_{t},b_{t})\nonumber \\ 203 | p(a_{t}|a_{t-1},b_{t}) & = & N(a_{t-1},b_{t})\label{eq:sys1}\\ 204 | p(b_{t}|b_{t-1}) & = & \log N(\log b_{t-1}-\frac{1}{2}\sigma_{b}^{2},\,\,\sigma_{b}^{2}),\sigma_{b}=0.01.\nonumber \end{eqnarray} 205 | 206 | \end_inset 207 | 208 | a jeho modifikaci 209 | \begin_inset Formula \begin{eqnarray*} 210 | p(x_{t}|x_{t-1}) & = & p(a_{t}|a_{t-1})p(b_{t}|b_{t-1},a_{t-1})\\ 211 | p(y_{t}|x_{t}) & = & p(y_{t}|b_{t})=N(a_{t},b_{t})\\ 212 | p(a_{t}|a_{t-1}) & = & N(a_{t-1},1)\\ 213 | p(b_{t}|b_{t-1},a_{t-1}) & = & \log N(\log b_{t-1}-\frac{1}{2}a_{t-1},a_{t-1}).\end{eqnarray*} 214 | 215 | \end_inset 216 | 217 | 218 | \end_layout 219 | 220 | \begin_layout Exercise 221 | Jak zadat hustotu 222 | \begin_inset Formula $p(x_{t}|x_{t-1})$ 223 | \end_inset 224 | 225 | pomoci CProdPdf. 226 | Jak bude vypadat vnitrek bayes(yt,[])? 227 | \end_layout 228 | 229 | \begin_layout Subsection* 230 | Marginalizovany PF 231 | \end_layout 232 | 233 | \begin_layout Standard 234 | Prvni system z predchozi casti lze odhadnout 235 | \begin_inset Quotes eld 236 | \end_inset 237 | 238 | chytreji 239 | \begin_inset Quotes erd 240 | \end_inset 241 | 242 | . 243 | V pripade, kdy bychom znali 244 | \begin_inset Formula $b_{t}$ 245 | \end_inset 246 | 247 | , jsou rovnice ( 248 | \begin_inset CommandInset ref 249 | LatexCommand ref 250 | reference "eq:sys1" 251 | 252 | \end_inset 253 | 254 | ) specialnim pripadem Kalmanova filtru pro 255 | \begin_inset Formula $A=1,B=0,C=1,D=0,Q=b_{t},R=b_{t}$ 256 | \end_inset 257 | 258 | . 259 | Aposteriorno na 260 | \begin_inset Formula $a$ 261 | \end_inset 262 | 263 | , 264 | \begin_inset Formula $p(a_{t}|b_{1:t},y_{1:t})$ 265 | \end_inset 266 | 267 | je tedy gausovka. 268 | 269 | \end_layout 270 | 271 | \begin_layout Standard 272 | To je zakladnim predpokladme marginalizovaneho particle filtru, ktery aproximuje 273 | aposteriorni hustotu 274 | \begin_inset Formula $p(a_{t},b_{t}|y_{1:t})$ 275 | \end_inset 276 | 277 | jako 278 | \begin_inset Formula \begin{eqnarray} 279 | p(a_{t},b_{t}|y_{1:t}) & = & \sum_{i=1}^{n}w_{i}p(a_{t}|b_{1:t},y_{1:t})\delta(b_{t}-b_{t}^{(i)})\label{eq:emp-1}\\ 280 | & = & \sum_{i=1}^{n}w_{i}N(\hat{a}_{t}^{(i)},P_{t}^{(i)})\delta(b_{t}-b_{t}^{(i)}),\end{eqnarray} 281 | 282 | \end_inset 283 | 284 | kde 285 | \begin_inset Formula $\hat{a}_{t}^{(i)},P_{t}^{(i)}$ 286 | \end_inset 287 | 288 | jsou statistiky Kalmanova filtru pro hodnoty 289 | \begin_inset Formula $b_{1:t}^{(i)}$ 290 | \end_inset 291 | 292 | 293 | \begin_inset Formula $i$ 294 | \end_inset 295 | 296 | -teho partiklu. 297 | \end_layout 298 | 299 | \begin_layout Standard 300 | Metoda bayes funguje nasledovne: 301 | \end_layout 302 | 303 | \begin_layout Enumerate 304 | nageneruj pocatecni 305 | \begin_inset Formula $b_{0}^{(1:n)}$ 306 | \end_inset 307 | 308 | , a nastav 309 | \begin_inset Formula $w_{i}=1/n,\forall i$ 310 | \end_inset 311 | 312 | 313 | \end_layout 314 | 315 | \begin_layout Enumerate 316 | pro kazde nove mereni 317 | \begin_inset Formula $y_{t}$ 318 | \end_inset 319 | 320 | 321 | \end_layout 322 | 323 | \begin_deeper 324 | \begin_layout Enumerate 325 | vygeneruj nove 326 | \begin_inset Formula $b_{t}^{(i)}$ 327 | \end_inset 328 | 329 | =sample from 330 | \begin_inset Formula $p(b_{t}^{(i)}|b_{t-1}^{(i)})$ 331 | \end_inset 332 | 333 | 334 | \end_layout 335 | 336 | \begin_layout Enumerate 337 | dosad nove 338 | \begin_inset Formula $b_{t}^{(i)}$ 339 | \end_inset 340 | 341 | do KF a proved KF.bayes(yt,[]). 342 | \end_layout 343 | 344 | \begin_layout Enumerate 345 | spocti marginalni verohodnost KF 346 | \family roman 347 | \series medium 348 | \shape up 349 | \size normal 350 | \emph off 351 | \bar no 352 | \noun off 353 | \color none 354 | 355 | \begin_inset Formula $p(y_{t}|y_{1:t},b_{1:t}^{(i)})=R_{y}^{-1/2}\exp(-\frac{1}{2}(\hat{a}_{t,t-1}-y_{t})R_{y}^{-1}(\hat{a}_{t,t-1}-y_{t}))$ 356 | \end_inset 357 | 358 | , 359 | \begin_inset Newline newline 360 | \end_inset 361 | 362 | 363 | \begin_inset Formula $\hat{a}_{t,t-1}$ 364 | \end_inset 365 | 366 | =KF[i].S.mu, 367 | \begin_inset Formula $R_{y}$ 368 | \end_inset 369 | 370 | = KF[i].S.R (pro kalmana - obecne bych rad, aby bayes() toto cislo vracel) 371 | \end_layout 372 | 373 | \begin_layout Enumerate 374 | prepocti 375 | \begin_inset Formula $w_{i}=p(y_{t}|y_{1:t},b_{t}^{(i)})w_{i},\forall i$ 376 | \end_inset 377 | 378 | 379 | \end_layout 380 | 381 | \begin_layout Enumerate 382 | normalizuj 383 | \begin_inset Formula $w=\frac{w}{\sum w}$ 384 | \end_inset 385 | 386 | 387 | \end_layout 388 | 389 | \begin_layout Enumerate 390 | resample 391 | \begin_inset Formula $w$ 392 | \end_inset 393 | 394 | , 395 | \begin_inset Formula $b_{t}$ 396 | \end_inset 397 | 398 | ,KF (viz. 399 | BDM::eEmp::resample) 400 | \end_layout 401 | 402 | \end_deeper 403 | \end_body 404 | \end_document 405 | --------------------------------------------------------------------------------