├── .coveragerc ├── .gitignore ├── AUTHORS.rst ├── CHANGES.rst ├── LICENSE.txt ├── README.rst ├── docs ├── Makefile ├── _static │ └── .gitignore ├── authors.rst ├── changes.rst ├── conf.py ├── index.rst └── license.rst ├── requirements.txt ├── setup.cfg ├── setup.py ├── test-requirements.txt ├── tests ├── compress │ └── test_compress.py └── conftest.py └── tspreprocess ├── __init__.py ├── compress ├── __init__.py └── compress.py ├── denoise ├── __init__.py └── denoise.py ├── resample ├── __init__.py └── resample.py └── transform ├── __init__.py └── transform.py /.coveragerc: -------------------------------------------------------------------------------- 1 | # .coveragerc to control coverage.py 2 | [run] 3 | branch = True 4 | source = tspreprocess 5 | # omit = bad_file.py 6 | 7 | [report] 8 | # Regexes for lines to exclude from consideration 9 | exclude_lines = 10 | # Have to re-enable the standard pragma 11 | pragma: no cover 12 | 13 | # Don't complain about missing debug-only code: 14 | def __repr__ 15 | if self\.debug 16 | 17 | # Don't complain if tests don't hit defensive assertion code: 18 | raise AssertionError 19 | raise NotImplementedError 20 | 21 | # Don't complain if non-runnable code isn't run: 22 | if 0: 23 | if __name__ == .__main__.: 24 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Temporary and binary files 2 | *~ 3 | *.py[cod] 4 | *.so 5 | *.cfg 6 | !setup.cfg 7 | *.orig 8 | *.log 9 | *.pot 10 | __pycache__/* 11 | .cache/* 12 | .*.swp 13 | */.ipynb_checkpoints/* 14 | 15 | # Project files 16 | .ropeproject 17 | .project 18 | .pydevproject 19 | .settings 20 | .idea 21 | 22 | # Package files 23 | *.egg 24 | *.eggs/ 25 | .installed.cfg 26 | *.egg-info 27 | 28 | # Unittest and coverage 29 | htmlcov/* 30 | .coverage 31 | .tox 32 | junit.xml 33 | coverage.xml 34 | 35 | # Build and docs folder/files 36 | build/* 37 | dist/* 38 | sdist/* 39 | docs/api/* 40 | docs/_build/* 41 | cover/* 42 | MANIFEST 43 | 44 | # Virtualenvironment 45 | /venv 46 | 47 | # Mac os folder files 48 | .DS_Store 49 | -------------------------------------------------------------------------------- /AUTHORS.rst: -------------------------------------------------------------------------------- 1 | ========== 2 | Developers 3 | ========== 4 | 5 | * Maximilian Christ 6 | -------------------------------------------------------------------------------- /CHANGES.rst: -------------------------------------------------------------------------------- 1 | ========= 2 | Changelog 3 | ========= 4 | 5 | Version 0.1 6 | =========== 7 | 8 | - Feature A added 9 | - FIX: nasty bug #1729 fixed 10 | - add your changes here! 11 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT LICENCE 2 | 3 | Copyright (c) 2017 Maximilian Christ 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated 6 | documentation files (the "Software"), to deal in the Software without restriction, including without limitation the 7 | rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit 8 | persons to whom the Software is furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the 11 | Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE 14 | WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR 15 | COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR 16 | OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | **Disclaimer: This package is WIP. Do not take any APIs for granted.** 2 | 3 | ============ 4 | tspreprocess 5 | ============ 6 | 7 | Time series can contain noise, may be sampled under a non fitting rate or just need to be compressed. 8 | *tspreprocess* is a library for such preprocessing tasks. It contains tools to transform and clean time series data for better analyses. 9 | 10 | In detail, we are planning to add methods to do 11 | 12 | * Denoising 13 | * Compression 14 | * Resampling 15 | * ... 16 | 17 | Our goal is to make this the most comprehensive time series preprocessing library. 18 | 19 | 20 | Installation 21 | ============ 22 | 23 | Clone the repo, cd into it and install it with pip locally 24 | 25 | .. code-block:: Python 26 | 27 | git clone https://github.com/MaxBenChrist/tspreprocess 28 | cd tspreprocess 29 | pip install -e . 30 | 31 | You can run the test suite by 32 | 33 | .. code-block:: Python 34 | 35 | python setup.py test 36 | 37 | 38 | Relation to *tsfresh* 39 | ===================== 40 | 41 | This package will based on the data formats from the python feature extraction pacakge *tsfresh* 42 | (https://github.com/blue-yonder/tsfresh), allowing a seamless integration between both packages. 43 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # User-friendly check for sphinx-build 11 | ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $?), 1) 12 | $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) 13 | endif 14 | 15 | # Internal variables. 16 | PAPEROPT_a4 = -D latex_paper_size=a4 17 | PAPEROPT_letter = -D latex_paper_size=letter 18 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 19 | # the i18n builder cannot share the environment and doctrees with the others 20 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 21 | 22 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 23 | 24 | help: 25 | @echo "Please use \`make ' where is one of" 26 | @echo " html to make standalone HTML files" 27 | @echo " dirhtml to make HTML files named index.html in directories" 28 | @echo " singlehtml to make a single large HTML file" 29 | @echo " pickle to make pickle files" 30 | @echo " json to make JSON files" 31 | @echo " htmlhelp to make HTML files and a HTML help project" 32 | @echo " qthelp to make HTML files and a qthelp project" 33 | @echo " devhelp to make HTML files and a Devhelp project" 34 | @echo " epub to make an epub" 35 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 36 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 37 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 38 | @echo " text to make text files" 39 | @echo " man to make manual pages" 40 | @echo " texinfo to make Texinfo files" 41 | @echo " info to make Texinfo files and run them through makeinfo" 42 | @echo " gettext to make PO message catalogs" 43 | @echo " changes to make an overview of all changed/added/deprecated items" 44 | @echo " xml to make Docutils-native XML files" 45 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 46 | @echo " linkcheck to check all external links for integrity" 47 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 48 | 49 | clean: 50 | rm -rf $(BUILDDIR)/* 51 | 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | dirhtml: 58 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 59 | @echo 60 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 61 | 62 | singlehtml: 63 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 64 | @echo 65 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 66 | 67 | pickle: 68 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 69 | @echo 70 | @echo "Build finished; now you can process the pickle files." 71 | 72 | json: 73 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 74 | @echo 75 | @echo "Build finished; now you can process the JSON files." 76 | 77 | htmlhelp: 78 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 79 | @echo 80 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 81 | ".hhp project file in $(BUILDDIR)/htmlhelp." 82 | 83 | qthelp: 84 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 85 | @echo 86 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 87 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 88 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/tspreprocess.qhcp" 89 | @echo "To view the help file:" 90 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/tspreprocess.qhc" 91 | 92 | devhelp: 93 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 94 | @echo 95 | @echo "Build finished." 96 | @echo "To view the help file:" 97 | @echo "# mkdir -p $HOME/.local/share/devhelp/tspreprocess" 98 | @echo "# ln -s $(BUILDDIR)/devhelp $HOME/.local/share/devhelp/tspreprocess" 99 | @echo "# devhelp" 100 | 101 | epub: 102 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 103 | @echo 104 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 105 | 106 | latex: 107 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 108 | @echo 109 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 110 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 111 | "(use \`make latexpdf' here to do that automatically)." 112 | 113 | latexpdf: 114 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 115 | @echo "Running LaTeX files through pdflatex..." 116 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 117 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 118 | 119 | latexpdfja: 120 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 121 | @echo "Running LaTeX files through platex and dvipdfmx..." 122 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 123 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 124 | 125 | text: 126 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 127 | @echo 128 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 129 | 130 | man: 131 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 132 | @echo 133 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 134 | 135 | texinfo: 136 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 137 | @echo 138 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 139 | @echo "Run \`make' in that directory to run these through makeinfo" \ 140 | "(use \`make info' here to do that automatically)." 141 | 142 | info: 143 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 144 | @echo "Running Texinfo files through makeinfo..." 145 | make -C $(BUILDDIR)/texinfo info 146 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 147 | 148 | gettext: 149 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 150 | @echo 151 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 152 | 153 | changes: 154 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 155 | @echo 156 | @echo "The overview file is in $(BUILDDIR)/changes." 157 | 158 | linkcheck: 159 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 160 | @echo 161 | @echo "Link check complete; look for any errors in the above output " \ 162 | "or in $(BUILDDIR)/linkcheck/output.txt." 163 | 164 | doctest: 165 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 166 | @echo "Testing of doctests in the sources finished, look at the " \ 167 | "results in $(BUILDDIR)/doctest/output.txt." 168 | 169 | xml: 170 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 171 | @echo 172 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 173 | 174 | pseudoxml: 175 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 176 | @echo 177 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." 178 | -------------------------------------------------------------------------------- /docs/_static/.gitignore: -------------------------------------------------------------------------------- 1 | # Empty directory 2 | -------------------------------------------------------------------------------- /docs/authors.rst: -------------------------------------------------------------------------------- 1 | .. _authors: 2 | .. include:: ../AUTHORS.rst 3 | -------------------------------------------------------------------------------- /docs/changes.rst: -------------------------------------------------------------------------------- 1 | .. _changes: 2 | .. include:: ../CHANGES.rst 3 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # 3 | # This file is execfile()d with the current directory set to its containing dir. 4 | # 5 | # Note that not all possible configuration values are present in this 6 | # autogenerated file. 7 | # 8 | # All configuration values have a default; values that are commented out 9 | # serve to show the default. 10 | 11 | import sys 12 | 13 | # If extensions (or modules to document with autodoc) are in another directory, 14 | # add these directories to sys.path here. If the directory is relative to the 15 | # documentation root, use os.path.abspath to make it absolute, like shown here. 16 | # sys.path.insert(0, os.path.abspath('.')) 17 | 18 | # -- Hack for ReadTheDocs ------------------------------------------------------ 19 | # This hack is necessary since RTD does not issue `sphinx-apidoc` before running 20 | # `sphinx-build -b html . _build/html`. See Issue: 21 | # https://github.com/rtfd/readthedocs.org/issues/1139 22 | # DON'T FORGET: Check the box "Install your project inside a virtualenv using 23 | # setup.py install" in the RTD Advanced Settings. 24 | import os 25 | on_rtd = os.environ.get('READTHEDOCS', None) == 'True' 26 | if on_rtd: 27 | import inspect 28 | from sphinx import apidoc 29 | 30 | __location__ = os.path.join(os.getcwd(), os.path.dirname( 31 | inspect.getfile(inspect.currentframe()))) 32 | 33 | output_dir = os.path.join(__location__, "../docs/api") 34 | module_dir = os.path.join(__location__, "../tspreprocess") 35 | cmd_line_template = "sphinx-apidoc -f -o {outputdir} {moduledir}" 36 | cmd_line = cmd_line_template.format(outputdir=output_dir, moduledir=module_dir) 37 | apidoc.main(cmd_line.split(" ")) 38 | 39 | # -- General configuration ----------------------------------------------------- 40 | 41 | # If your documentation needs a minimal Sphinx version, state it here. 42 | # needs_sphinx = '1.0' 43 | 44 | # Add any Sphinx extension module names here, as strings. They can be extensions 45 | # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 46 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx', 'sphinx.ext.todo', 47 | 'sphinx.ext.autosummary', 'sphinx.ext.viewcode', 'sphinx.ext.coverage', 48 | 'sphinx.ext.doctest', 'sphinx.ext.ifconfig', 'sphinx.ext.pngmath', 49 | 'sphinx.ext.napoleon'] 50 | 51 | # Add any paths that contain templates here, relative to this directory. 52 | templates_path = ['_templates'] 53 | 54 | # The suffix of source filenames. 55 | source_suffix = '.rst' 56 | 57 | # The encoding of source files. 58 | # source_encoding = 'utf-8-sig' 59 | 60 | # The master toctree document. 61 | master_doc = 'index' 62 | 63 | # General information about the project. 64 | project = u'tspreprocess' 65 | copyright = u'2017, Maximilian Christ' 66 | 67 | # The version info for the project you're documenting, acts as replacement for 68 | # |version| and |release|, also used in various other places throughout the 69 | # built documents. 70 | # 71 | # The short X.Y version. 72 | version = '' # Is set by calling `setup.py docs` 73 | # The full version, including alpha/beta/rc tags. 74 | release = '' # Is set by calling `setup.py docs` 75 | 76 | # The language for content autogenerated by Sphinx. Refer to documentation 77 | # for a list of supported languages. 78 | # language = None 79 | 80 | # There are two options for replacing |today|: either, you set today to some 81 | # non-false value, then it is used: 82 | # today = '' 83 | # Else, today_fmt is used as the format for a strftime call. 84 | # today_fmt = '%B %d, %Y' 85 | 86 | # List of patterns, relative to source directory, that match files and 87 | # directories to ignore when looking for source files. 88 | exclude_patterns = ['_build'] 89 | 90 | # The reST default role (used for this markup: `text`) to use for all documents. 91 | # default_role = None 92 | 93 | # If true, '()' will be appended to :func: etc. cross-reference text. 94 | # add_function_parentheses = True 95 | 96 | # If true, the current module name will be prepended to all description 97 | # unit titles (such as .. function::). 98 | # add_module_names = True 99 | 100 | # If true, sectionauthor and moduleauthor directives will be shown in the 101 | # output. They are ignored by default. 102 | # show_authors = False 103 | 104 | # The name of the Pygments (syntax highlighting) style to use. 105 | pygments_style = 'sphinx' 106 | 107 | # A list of ignored prefixes for module index sorting. 108 | # modindex_common_prefix = [] 109 | 110 | # If true, keep warnings as "system message" paragraphs in the built documents. 111 | # keep_warnings = False 112 | 113 | 114 | # -- Options for HTML output --------------------------------------------------- 115 | 116 | # The theme to use for HTML and HTML Help pages. See the documentation for 117 | # a list of builtin themes. 118 | html_theme = 'alabaster' 119 | 120 | # Theme options are theme-specific and customize the look and feel of a theme 121 | # further. For a list of options available for each theme, see the 122 | # documentation. 123 | # html_theme_options = {} 124 | 125 | # Add any paths that contain custom themes here, relative to this directory. 126 | # html_theme_path = [] 127 | 128 | # The name for this set of Sphinx documents. If None, it defaults to 129 | # " v documentation". 130 | try: 131 | from tspreprocess import __version__ as version 132 | except ImportError: 133 | pass 134 | else: 135 | release = version 136 | 137 | # A shorter title for the navigation bar. Default is the same as html_title. 138 | # html_short_title = None 139 | 140 | # The name of an image file (relative to this directory) to place at the top 141 | # of the sidebar. 142 | # html_logo = "" 143 | 144 | # The name of an image file (within the static path) to use as favicon of the 145 | # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 146 | # pixels large. 147 | # html_favicon = None 148 | 149 | # Add any paths that contain custom static files (such as style sheets) here, 150 | # relative to this directory. They are copied after the builtin static files, 151 | # so a file named "default.css" will overwrite the builtin "default.css". 152 | html_static_path = ['_static'] 153 | 154 | # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, 155 | # using the given strftime format. 156 | # html_last_updated_fmt = '%b %d, %Y' 157 | 158 | # If true, SmartyPants will be used to convert quotes and dashes to 159 | # typographically correct entities. 160 | # html_use_smartypants = True 161 | 162 | # Custom sidebar templates, maps document names to template names. 163 | # html_sidebars = {} 164 | 165 | # Additional templates that should be rendered to pages, maps page names to 166 | # template names. 167 | # html_additional_pages = {} 168 | 169 | # If false, no module index is generated. 170 | # html_domain_indices = True 171 | 172 | # If false, no index is generated. 173 | # html_use_index = True 174 | 175 | # If true, the index is split into individual pages for each letter. 176 | # html_split_index = False 177 | 178 | # If true, links to the reST sources are added to the pages. 179 | # html_show_sourcelink = True 180 | 181 | # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. 182 | # html_show_sphinx = True 183 | 184 | # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. 185 | # html_show_copyright = True 186 | 187 | # If true, an OpenSearch description file will be output, and all pages will 188 | # contain a tag referring to it. The value of this option must be the 189 | # base URL from which the finished HTML is served. 190 | # html_use_opensearch = '' 191 | 192 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 193 | # html_file_suffix = None 194 | 195 | # Output file base name for HTML help builder. 196 | htmlhelp_basename = 'tspreprocess-doc' 197 | 198 | 199 | # -- Options for LaTeX output -------------------------------------------------- 200 | 201 | latex_elements = { 202 | # The paper size ('letterpaper' or 'a4paper'). 203 | # 'papersize': 'letterpaper', 204 | 205 | # The font size ('10pt', '11pt' or '12pt'). 206 | # 'pointsize': '10pt', 207 | 208 | # Additional stuff for the LaTeX preamble. 209 | # 'preamble': '', 210 | } 211 | 212 | # Grouping the document tree into LaTeX files. List of tuples 213 | # (source start file, target name, title, author, documentclass [howto/manual]). 214 | latex_documents = [ 215 | ('index', 'user_guide.tex', u'tspreprocess Documentation', 216 | u'Maximilian Christ', 'manual'), 217 | ] 218 | 219 | # The name of an image file (relative to this directory) to place at the top of 220 | # the title page. 221 | # latex_logo = "" 222 | 223 | # For "manual" documents, if this is true, then toplevel headings are parts, 224 | # not chapters. 225 | # latex_use_parts = False 226 | 227 | # If true, show page references after internal links. 228 | # latex_show_pagerefs = False 229 | 230 | # If true, show URL addresses after external links. 231 | # latex_show_urls = False 232 | 233 | # Documents to append as an appendix to all manuals. 234 | # latex_appendices = [] 235 | 236 | # If false, no module index is generated. 237 | # latex_domain_indices = True 238 | 239 | # -- External mapping ------------------------------------------------------------ 240 | python_version = '.'.join(map(str, sys.version_info[0:2])) 241 | intersphinx_mapping = { 242 | 'sphinx': ('http://sphinx.pocoo.org', None), 243 | 'python': ('http://docs.python.org/' + python_version, None), 244 | 'matplotlib': ('http://matplotlib.sourceforge.net', None), 245 | 'numpy': ('http://docs.scipy.org/doc/numpy', None), 246 | 'sklearn': ('http://scikit-learn.org/stable', None), 247 | 'pandas': ('http://pandas.pydata.org/pandas-docs/stable', None), 248 | 'scipy': ('http://docs.scipy.org/doc/scipy/reference/', None), 249 | } 250 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | tspreprocess 3 | ============ 4 | 5 | This is the documentation of **tspreprocess**. 6 | 7 | .. note:: 8 | 9 | This is the main page of your project's `Sphinx `_ 10 | documentation. It is formatted in `reStructuredText 11 | `__. Add additional pages by creating 12 | rst-files in ``docs`` and adding them to the `toctree 13 | `_ below. Use then 14 | `references `__ in order to link 15 | them from this page, e.g. :ref:`authors ` and :ref:`changes`. 16 | 17 | It is also possible to refer to the documentation of other Python packages 18 | with the `Python domain syntax 19 | `__. By default you 20 | can reference the documentation of `Sphinx `__, 21 | `Python `__, `NumPy 22 | `__, `SciPy 23 | `__, `matplotlib 24 | `__, `Pandas 25 | `__, `Scikit-Learn 26 | `__. You can add more by 27 | extending the ``intersphinx_mapping`` in your Sphinx's ``conf.py``. 28 | 29 | The pretty useful extension `autodoc 30 | `__ is activated by 31 | default and lets you include documentation from docstrings. Docstrings can 32 | be written in `Google 33 | `__ 34 | (recommended!), `NumPy 35 | `__ 36 | and `classical 37 | `__ 38 | style. 39 | 40 | 41 | Contents 42 | ======== 43 | 44 | .. toctree:: 45 | :maxdepth: 2 46 | 47 | License 48 | Authors 49 | Changelog 50 | Module Reference 51 | 52 | 53 | Indices and tables 54 | ================== 55 | 56 | * :ref:`genindex` 57 | * :ref:`modindex` 58 | * :ref:`search` 59 | -------------------------------------------------------------------------------- /docs/license.rst: -------------------------------------------------------------------------------- 1 | .. _license: 2 | 3 | ======= 4 | License 5 | ======= 6 | 7 | .. literalinclude:: ../LICENSE.txt 8 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | # Add your requirements here like: 2 | numpy==1.13.1 3 | pandas==0.20.3 4 | six==1.10.0 5 | tsfresh==0.9.0 6 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | name = tspreprocess 3 | summary = Add a short description here! 4 | author = Maximilian Christ 5 | author-email = max.christ@me.com 6 | license = none 7 | home-page = http://... 8 | description-file = README.rst 9 | # Add here all kinds of additional classifiers as defined under 10 | # https://pypi.python.org/pypi?%3Aaction=list_classifiers 11 | classifier = 12 | Development Status :: 4 - Beta 13 | Programming Language :: Python 14 | 15 | [entry_points] 16 | # Add here console scripts like: 17 | # console_scripts = 18 | # script_name = tspreprocess.module:function 19 | # For example: 20 | # console_scripts = 21 | # fibonacci = tspreprocess.skeleton:run 22 | # as well as other entry_points. 23 | 24 | 25 | [files] 26 | # Add here 'data_files', 'packages' or 'namespace_packages'. 27 | # Additional data files are defined as key value pairs of target directory 28 | # and source location from the root of the repository: 29 | packages = 30 | tspreprocess 31 | # data_files = 32 | # share/tspreprocess_docs = docs/* 33 | 34 | [extras] 35 | # Add here additional requirements for extra features, like: 36 | # PDF = 37 | # ReportLab>=1.2 38 | # RXP 39 | 40 | [test] 41 | # py.test options when running `python setup.py test` 42 | addopts = tests 43 | 44 | [tool:pytest] 45 | # Options for py.test: 46 | # Specify command line options as you would do when invoking py.test directly. 47 | # e.g. --cov-report html (or xml) for html/xml output or --junitxml junit.xml 48 | # in order to write a coverage file that can be read by Jenkins. 49 | addopts = 50 | --cov tspreprocess --cov-report term-missing 51 | --verbose 52 | 53 | [aliases] 54 | docs = build_sphinx 55 | 56 | [bdist_wheel] 57 | # Use this option if your package is pure-python 58 | universal = 1 59 | 60 | [build_sphinx] 61 | source_dir = docs 62 | build_dir = docs/_build 63 | 64 | [pbr] 65 | # Let pbr run sphinx-apidoc 66 | autodoc_tree_index_modules = True 67 | # autodoc_tree_excludes = ... 68 | # Let pbr itself generate the apidoc 69 | # autodoc_index_modules = True 70 | # autodoc_exclude_modules = ... 71 | # Convert warnings to errors 72 | # warnerrors = True 73 | 74 | [devpi:upload] 75 | # Options for the devpi: PyPI server and packaging tool 76 | # VCS export must be deactivated since we are using setuptools-scm 77 | no-vcs = 1 78 | formats = bdist_wheel 79 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Setup file for tspreprocess. 5 | 6 | This file was generated with PyScaffold 2.5.7, a tool that easily 7 | puts up a scaffold for your new Python project. Learn more under: 8 | http://pyscaffold.readthedocs.org/ 9 | """ 10 | 11 | import sys 12 | from setuptools import setup 13 | 14 | 15 | def setup_package(): 16 | needs_sphinx = {'build_sphinx', 'upload_docs'}.intersection(sys.argv) 17 | sphinx = ['sphinx'] if needs_sphinx else [] 18 | setup(setup_requires=['six', 'pyscaffold>=2.5a0,<2.6a0'] + sphinx, 19 | use_pyscaffold=True) 20 | 21 | 22 | if __name__ == "__main__": 23 | setup_package() 24 | -------------------------------------------------------------------------------- /test-requirements.txt: -------------------------------------------------------------------------------- 1 | # Add requirements only needed for your unittests and during development here. 2 | # They will be installed automatically when running `python setup.py test`. 3 | # ATTENTION: Don't remove pytest-cov and pytest as they are needed. 4 | pytest-cov 5 | pytest 6 | -------------------------------------------------------------------------------- /tests/compress/test_compress.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # This file as well as the whole tspreprocess package are licenced under the MIT licence (see the LICENCE.txt) 3 | # Maximilian Christ (maximilianchrist.com), 2017 4 | 5 | from __future__ import absolute_import, division 6 | import numpy as np 7 | import pandas as pd 8 | from pandas.testing import assert_frame_equal 9 | from tspreprocess.compress.compress import compress 10 | from unittest import TestCase 11 | 12 | 13 | class CompressTestCase(TestCase): 14 | def setUp(self): 15 | cid = np.repeat([10, 500], 10) 16 | ckind = np.repeat(["a", "b", "a", "b"], 5) 17 | csort = [1, 2, 3, 4, 5, 18 | 6, 7, 8, 9, 10, 19 | 1, 2, 3, 4, 5, 20 | 6, 7, 8, 9, 10] 21 | cval = [11, 9, 67, 45, 30, 22 | 58, 62, 19, 56, 29, 23 | 0, 27, 36, 43, 33, 24 | 2, 24, 71, 41, 28] 25 | self.df = pd.DataFrame({"id": cid, "kind": ckind, "sort": csort, "val": cval}) 26 | self.col_naming = {"column_id": "id", "column_kind": "kind", "column_sort": "sort", "column_value": "val"} 27 | 28 | # gives the following DataFrame 29 | """ 30 | id kind sort val 31 | 0 10 a 1 11 32 | 1 10 a 2 9 33 | 2 10 a 3 67 34 | 3 10 a 4 45 35 | 4 10 a 5 30 36 | 5 10 b 6 58 37 | 6 10 b 7 62 38 | 7 10 b 8 19 39 | 8 10 b 9 56 40 | 9 10 b 10 29 41 | 10 500 a 1 0 42 | 11 500 a 2 27 43 | 12 500 a 3 36 44 | 13 500 a 4 43 45 | 14 500 a 5 33 46 | 15 500 b 6 2 47 | 16 500 b 7 24 48 | 17 500 b 8 71 49 | 18 500 b 9 41 50 | 19 500 b 10 28 51 | """ 52 | 53 | def test_compress_max(self): 54 | dd = compress(self.df, 55 | compression_functions={"maximum": None}, 56 | interval_length=2, 57 | **self.col_naming) 58 | 59 | expected_dd = pd.DataFrame({"id": ["10"] * 3 + ["500"] * 3, 60 | "sort": ["bin_0", "bin_1", "bin_2"] * 2, 61 | "a_maximum": [11., 67., 30., 27., 43., 33.], 62 | "b_maximum": [62., 56., 29., 24., 71., 28.]}) 63 | 64 | expected_dd = expected_dd[dd.columns] 65 | 66 | expected_dd.sort_values(by=["id", "sort"], inplace=True) 67 | assert_frame_equal(expected_dd, dd) 68 | 69 | def test_compress_min(self): 70 | dd = compress(self.df, 71 | compression_functions={"minimum": None}, 72 | interval_length=2, 73 | **self.col_naming) 74 | 75 | expected_dd = pd.DataFrame({"id": ["10"] * 3 + ["500"] * 3, 76 | "sort": ["bin_0", "bin_1", "bin_2"] * 2, 77 | "a_minimum": [9., 45., 30., 0., 36., 33.], 78 | "b_minimum": [58., 19., 29., 2., 41., 28.]}) 79 | 80 | expected_dd = expected_dd[dd.columns] 81 | expected_dd.sort_values(by=["id", "sort"], inplace=True) 82 | assert_frame_equal(expected_dd, dd) 83 | 84 | 85 | # todo: we need tests for the other tsfresh formats, maybe just restructure the DF from above 86 | -------------------------------------------------------------------------------- /tests/conftest.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | """ 4 | Dummy conftest.py for tspreprocess. 5 | 6 | If you don't know what this is for, just leave it empty. 7 | Read more about conftest.py under: 8 | https://pytest.org/latest/plugins.html 9 | """ 10 | from __future__ import print_function, absolute_import, division 11 | 12 | import pytest 13 | -------------------------------------------------------------------------------- /tspreprocess/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | import pkg_resources 3 | 4 | try: 5 | __version__ = pkg_resources.get_distribution(__name__).version 6 | except: 7 | __version__ = 'unknown' 8 | -------------------------------------------------------------------------------- /tspreprocess/compress/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MaxBenChrist/tspreprocess/08695c287227855732e3d2213134f9e65649ac70/tspreprocess/compress/__init__.py -------------------------------------------------------------------------------- /tspreprocess/compress/compress.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # This file as well as the whole tspreprocess package are licenced under the MIT licence (see the LICENCE.txt) 3 | # Maximilian Christ (maximilianchrist.com), 2017 4 | 5 | from __future__ import absolute_import, division 6 | import numpy as np 7 | from tsfresh import extract_features 8 | from tsfresh.utilities.dataframe_functions import _normalize_input_to_internal_representation 9 | 10 | 11 | # todo: maybe compress is not the right term? because we are losing information 12 | # todo: add support for generic numpy methods as aggregation function 13 | # todo: add support for custom aggregation functions 14 | def compress(ts, compression_functions, interval_length, column_id, column_sort, column_kind, column_value): 15 | """ 16 | This method compresses time series by applying a compression function on bins. Then the values of the compression 17 | function over the bins are returned as a new, compressed time series. 18 | 19 | This decreasing the memory footprint of the time series. E.g. by applying a singular compression function on chunks 20 | of size 10, the time series is compressed by a factor 10. 21 | 22 | It is also possible to use multiple compression functions. 23 | 24 | The time series container ts must be in one of the formats that are supported by the tsfresh package. 25 | 26 | :param ts: The pandas.DataFrame with the time series to compute the features for, or a dictionary of pandas.DataFrames. 27 | :type ts: pandas.DataFrame or dict 28 | 29 | :param compression_functions: mapping from feature calculator names to parameters. See tsfresh documentation 30 | :type compression_functions: dict 31 | 32 | :param interval_length: the length of each bin to which the aggregation functions are applied 33 | :type interval_length: int 34 | 35 | :param column_id: The name of the id column to group by. 36 | :type column_id: str 37 | 38 | :param column_sort: The name of the sort column. 39 | :type column_sort: str 40 | 41 | :param column_kind: The name of the column keeping record on the kind of the value. 42 | :type column_kind: str 43 | 44 | :param column_value: The name for the column keeping the value itself. 45 | :type column_value: str 46 | """ 47 | 48 | dd, column_id, column_kind, column_value = \ 49 | _normalize_input_to_internal_representation(ts, column_id, column_sort, column_kind, column_value) 50 | 51 | def create_bins(v): 52 | n_bins = np.ceil(len(v) / interval_length) 53 | return np.repeat(np.arange(n_bins), interval_length)[:len(v)] 54 | 55 | dd[column_id] = dd[column_id].apply(str) + "_bin_" + \ 56 | dd.groupby([column_id, column_kind])[column_value].transform(create_bins).apply(str) 57 | 58 | dd = extract_features(dd, 59 | column_id=column_id, 60 | column_value=column_value, 61 | column_kind=column_kind, 62 | default_fc_parameters=compression_functions) 63 | 64 | dd.columns = [x.replace("__", "_") for x in dd.columns] 65 | dd.columns = [x.replace("feature", "map") for x in dd.columns] 66 | dd.reset_index(drop=False, inplace=True) 67 | 68 | ids = dd[column_id].str.split("_bin_").apply(lambda s: s[0]) 69 | bin_number = dd["id"].str.split("_bin_").apply(lambda s: eval(s[1])) 70 | 71 | dd[column_id] = ids 72 | dd["bin"] = bin_number 73 | 74 | return dd.sort_values(by=[column_id, "bin"]) 75 | 76 | 77 | # todo: add references to SCADA sources 78 | # add unit tests for this method 79 | def compress_SCADA_like(ts, interval_length, column_id, column_sort, column_kind, column_value): 80 | """ 81 | Takes a tsfresh compatible time series container and performs compression by calculating max, min, mean and 82 | variance of each time series. 83 | 84 | This is a common compression technique for SCADA (Supervisory Control and Data Acquisition) systems, deployed in 85 | Industrial environments. 86 | 87 | """ 88 | return compress(ts=ts, 89 | compression_functions={"minimum": None, "maximum": None, "mean": None, "variance": None}, 90 | intervall_lenght=interval_length, 91 | column_id=column_id, column_sort=column_sort, column_kind=column_kind, column_value=column_value) 92 | 93 | 94 | def make_SAX(): 95 | 96 | pass -------------------------------------------------------------------------------- /tspreprocess/denoise/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MaxBenChrist/tspreprocess/08695c287227855732e3d2213134f9e65649ac70/tspreprocess/denoise/__init__.py -------------------------------------------------------------------------------- /tspreprocess/denoise/denoise.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # This file as well as the whole tspreprocess package are licenced under the MIT licence (see the LICENCE.txt) 3 | # Maximilian Christ (maximilianchrist.com), 2017 4 | 5 | from __future__ import absolute_import, division 6 | 7 | 8 | def make_stationary(): 9 | """ 10 | Takes a time series container ts and makes sure that all time series are stationary, 11 | """ 12 | pass 13 | 14 | 15 | def remove_outlier_sigma(): 16 | """ 17 | Removes outlier that are more than r sigma from the normal distribution 18 | """ 19 | pass 20 | 21 | 22 | def apply_kalman_filtering(): 23 | """ 24 | Applys a kalman filter to the time series. 25 | """ 26 | # todo: check https://github.com/pykalman/pykalman 27 | # todo: maybe this is more transformation than denoising 28 | pass 29 | 30 | 31 | -------------------------------------------------------------------------------- /tspreprocess/resample/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MaxBenChrist/tspreprocess/08695c287227855732e3d2213134f9e65649ac70/tspreprocess/resample/__init__.py -------------------------------------------------------------------------------- /tspreprocess/resample/resample.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # This file as well as the whole tspreprocess package are licenced under the MIT licence (see the LICENCE.txt) 3 | # Maximilian Christ (maximilianchrist.com), 2017 4 | 5 | from __future__ import absolute_import, division 6 | 7 | 8 | def resample_ts_container(): 9 | """ 10 | This method takes a time series container. Then it resamples every time series, while following the 11 | pandas.resample syntax 12 | """ 13 | pass -------------------------------------------------------------------------------- /tspreprocess/transform/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MaxBenChrist/tspreprocess/08695c287227855732e3d2213134f9e65649ac70/tspreprocess/transform/__init__.py -------------------------------------------------------------------------------- /tspreprocess/transform/transform.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | # This file as well as the whole tspreprocess package are licenced under the MIT licence (see the LICENCE.txt) 3 | # Maximilian Christ (maximilianchrist.com), 2017 4 | 5 | 6 | def create_SAX(): 7 | """ 8 | Takes a time series container and transforms the contained time series into SAX format 9 | """ 10 | # todo: check https://github.com/nphoff/saxpy 11 | pass 12 | 13 | 14 | def transform_to_splines(): 15 | """ 16 | First, fits a spline on the time series data. Then replaces the time series by the interpolated spline values. 17 | :return: 18 | """ 19 | pass --------------------------------------------------------------------------------