├── .gitignore ├── AUTHORS.rst ├── CONTRIBUTING.rst ├── HISTORY.rst ├── LICENSE ├── MANIFEST.in ├── README.rst ├── docs ├── Makefile ├── authors.rst ├── conf.py ├── contributing.rst ├── history.rst ├── index.rst ├── installation.rst ├── make.bat ├── readme.rst └── usage.rst ├── requirements.txt ├── scripts ├── render_access2xls └── render_access2xls_gui ├── setup.py └── xlrenderer ├── __init__.py ├── utils.py └── xlrenderer.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.py[cod] 2 | 3 | # C extensions 4 | *.so 5 | 6 | # Packages 7 | __pycache__ 8 | *.egg 9 | *.egg-info 10 | dist 11 | build 12 | eggs 13 | parts 14 | bin 15 | var 16 | sdist 17 | develop-eggs 18 | .installed.cfg 19 | lib 20 | lib64 21 | 22 | # Installer logs 23 | pip-log.txt 24 | 25 | # Unit test / coverage reports 26 | .coverage 27 | .tox 28 | nosetests.xml 29 | htmlcov 30 | 31 | # Translations 32 | *.mo 33 | 34 | # Mr Developer 35 | .mr.developer.cfg 36 | .project 37 | .pydevproject 38 | 39 | # Complexity 40 | output/*.html 41 | output/*/index.html 42 | 43 | # Sphinx 44 | docs/_build 45 | -------------------------------------------------------------------------------- /AUTHORS.rst: -------------------------------------------------------------------------------- 1 | ======= 2 | Credits 3 | ======= 4 | 5 | Development Lead 6 | ---------------- 7 | 8 | * Benoit Bovy 9 | 10 | Contributors 11 | ------------ 12 | 13 | None yet. Why not be the first? 14 | -------------------------------------------------------------------------------- /CONTRIBUTING.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | Contributing 3 | ============ 4 | 5 | Contributions are welcome, and they are greatly appreciated! Every 6 | little bit helps, and credit will always be given. 7 | 8 | You can contribute in many ways: 9 | 10 | Types of Contributions 11 | ---------------------- 12 | 13 | Report Bugs 14 | ~~~~~~~~~~~ 15 | 16 | Report bugs at https://github.com/benbovy/xlrenderer/issues. 17 | 18 | If you are reporting a bug, please include: 19 | 20 | * Your operating system name and version. 21 | * Any details about your local setup that might be helpful in troubleshooting. 22 | * Detailed steps to reproduce the bug. 23 | 24 | Fix Bugs 25 | ~~~~~~~~ 26 | 27 | Look through the GitHub issues for bugs. Anything tagged with "bug" 28 | is open to whoever wants to implement it. 29 | 30 | Implement Features 31 | ~~~~~~~~~~~~~~~~~~ 32 | 33 | Look through the GitHub issues for features. Anything tagged with "feature" 34 | is open to whoever wants to implement it. 35 | 36 | Write Documentation 37 | ~~~~~~~~~~~~~~~~~~~ 38 | 39 | xlrenderer could always use more documentation, whether as part of the 40 | official xlrenderer docs, in docstrings, or even on the web in blog posts, 41 | articles, and such. 42 | 43 | Submit Feedback 44 | ~~~~~~~~~~~~~~~ 45 | 46 | The best way to send feedback is to file an issue at https://github.com/benbovy/xlrenderer/issues. 47 | 48 | If you are proposing a feature: 49 | 50 | * Explain in detail how it would work. 51 | * Keep the scope as narrow as possible, to make it easier to implement. 52 | * Remember that this is a volunteer-driven project, and that contributions 53 | are welcome :) 54 | 55 | Get Started! 56 | ------------ 57 | 58 | Ready to contribute? Here's how to set up `xlrenderer` for local development. 59 | 60 | 1. Fork the `xlrenderer` repo on GitHub. 61 | 2. Clone your fork locally:: 62 | 63 | $ git clone git@github.com:your_name_here/xlrenderer.git 64 | 65 | 3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:: 66 | 67 | $ mkvirtualenv xlrenderer 68 | $ cd xlrenderer/ 69 | $ python setup.py develop 70 | 71 | 4. Create a branch for local development:: 72 | 73 | $ git checkout -b name-of-your-bugfix-or-feature 74 | 75 | Now you can make your changes locally. 76 | 77 | 5. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox:: 78 | 79 | $ flake8 xlrenderer tests 80 | $ python setup.py test 81 | $ tox 82 | 83 | To get flake8 and tox, just pip install them into your virtualenv. 84 | 85 | 6. Commit your changes and push your branch to GitHub:: 86 | 87 | $ git add . 88 | $ git commit -m "Your detailed description of your changes." 89 | $ git push origin name-of-your-bugfix-or-feature 90 | 91 | 7. Submit a pull request through the GitHub website. 92 | 93 | Pull Request Guidelines 94 | ----------------------- 95 | 96 | Before you submit a pull request, check that it meets these guidelines: 97 | 98 | 1. The pull request should include tests. 99 | 2. If the pull request adds functionality, the docs should be updated. Put 100 | your new functionality into a function with a docstring, and add the 101 | feature to the list in README.rst. 102 | 3. The pull request should work for Python 2.6, 2.7, 3.3, and 3.4, and for PyPy. Check 103 | https://travis-ci.org/benbovy/xlrenderer/pull_requests 104 | and make sure that the tests pass for all supported Python versions. 105 | 106 | Tips 107 | ---- 108 | 109 | To run a subset of tests:: 110 | 111 | $ python -m unittest tests.test_xlrenderer 112 | -------------------------------------------------------------------------------- /HISTORY.rst: -------------------------------------------------------------------------------- 1 | .. :changelog: 2 | 3 | History 4 | ------- 5 | 6 | 0.1.0 (2015-09-25) 7 | --------------------- 8 | 9 | * First release on PyPI. 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Benoit Bovy 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include AUTHORS.rst 2 | include CONTRIBUTING.rst 3 | include HISTORY.rst 4 | include LICENSE 5 | include README.rst 6 | 7 | recursive-include tests * 8 | recursive-exclude * __pycache__ 9 | recursive-exclude * *.py[co] 10 | 11 | recursive-include docs *.rst conf.py Makefile make.bat 12 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | =============================== 2 | xlrenderer 3 | =============================== 4 | 5 | Populate and render Excel templates from any database, using a single 6 | YAML definition file. 7 | 8 | Features 9 | -------- 10 | 11 | Given any database supported by `sqlalchemy` and a "template" Excel file, this 12 | package allows to generate (many) data-populated Excel files (and PDFs, 13 | Windows only), according to user-defined queries and worksheet cell 14 | location/content specified in a definition file. 15 | 16 | The structure and format (YAML) of the definition file allow to render 17 | complex templates with little effort. The jinja2 templating language is used 18 | for easy content rendering. 19 | 20 | Installation 21 | ------------ 22 | 23 | This package is only available for Windows and OSX platforms with 24 | Excel installed. 25 | 26 | Given that all requirements below are satisfied, run: 27 | 28 | $ python setup.py install 29 | 30 | No package available yet on PyPI or Anaconda.org. 31 | 32 | Requirements 33 | ------------ 34 | 35 | - pandas 36 | - sqlalchemy 37 | - xlwings 38 | - pyyaml 39 | - jinja2 40 | 41 | Usage 42 | ----- 43 | 44 | Basic usage from within Python: 45 | 46 | .. code-block:: python 47 | 48 | >>> from xlrenderer import ExcelTemplateRenderer 49 | >>> from sqlalchemy import create_engine 50 | >>> engine = create_engine('protocol://user@localhost:port/mydatabase') 51 | >>> xltemplate = "/path/to/excel/template.xlsx" 52 | >>> def_file = "/path/to/database2excel.yml" 53 | >>> outdir = "/path/to/outputdir" 54 | >>> r = ExcelTemplateRenderer(engine, xltemplate, def_file, outdir) 55 | >>> r.render() 56 | 57 | Taking the contacts table below as an example (stored in a relational database), 58 | 59 | +----+------------+-----------+------------+--------+ 60 | | id | first_name | last_name | birth_date | gender | 61 | +====+============+===========+============+========+ 62 | | 1 | Anna | Harper | 04-03-1982 | F | 63 | +----+------------+-----------+------------+--------+ 64 | | 2 | Fred | Lloyd | 10-12-1976 | M | 65 | +----+------------+-----------+------------+--------+ 66 | | 3 | John | Doe | 22-06-1965 | M | 67 | +----+------------+-----------+------------+--------+ 68 | | 4 | Daisy | Schaefer | 08-09-1989 | F | 69 | +----+------------+-----------+------------+--------+ 70 | 71 | A simple YAML definition block would look like 72 | 73 | .. code-block:: yaml 74 | 75 | - name: simple contact table 76 | query: > 77 | SELECT * FROM [CONTACTS] 78 | apply_by_row: no 79 | cell_specification: 80 | worksheet: "Contacts" 81 | top_left_cell: A1 82 | header: yes 83 | index: no 84 | save_as: 85 | filename: "contacts.xlsx" 86 | export_pdf: yes 87 | 88 | where ``name`` is any name given to the definition block (see below), ``query`` is the SQL query used to get data from the database and render it in the Excel template, ``apply_by_row: no`` here means that the whole query result will be rendered as a table of contiguous cells in the xls file, and the ``cell_specification`` block is where we define the name of the worksheet, the top-left cell of the rendered table and whether or not to show the header (i.e., field names) and the index (here the ``id`` key). Finally, the ``save_as`` block allows to save the rendered template in a separate file, with an option to also export it as PDF. 89 | 90 | More advanced rendering is possible. For example, the template might here consist of a custom contact form (non-contiguous cells) to be filled and rendered for each person in separate files. The corresponding YAML definition block would then look like 91 | 92 | .. code-block:: yaml 93 | 94 | - name: custom contact form 95 | query: > 96 | SELECT * FROM [CONTACTS] 97 | apply_by_row: yes 98 | cell_specification: 99 | worksheet: "Contact Info" 100 | cells: 101 | - { cell: B2, content: "{{ first_name|capitalize }}" } 102 | - { cell: B3, content: "{{ last_name|capitalize }}" } 103 | - { cell: C6, content: "{{ birth_date.strftime('%d/%m/%Y') }}" } 104 | - { cell: E6, content: "{% if gender == 'M' %}X{% endif %}" } 105 | - { cell: E7, content: "{% if gender == 'F' %}X{% endif %}" } 106 | save_as: 107 | filename: "{{ first_name }}-{{ last_name }}.xlsx" 108 | export_pdf: yes 109 | 110 | Note ``apply_by_row: yes`` which will fill, render and export the template for each row of the query result. Note also the use of jinja2's templating language for the cell content and filename. 111 | 112 | For even more advanced rendering, it is possible to combine multiple definition blocks (with results from different queries) using ``include``, e.g., 113 | 114 | .. code-block:: yaml 115 | 116 | - name: custom contact form 117 | query: > 118 | SELECT * FROM [CONTACTS] 119 | include: 120 | - name of another definition block 121 | save_as: 122 | filename: "{{ first_name }}-{{ last_name }}.xlsx" 123 | export_pdf: yes 124 | 125 | License 126 | ------- 127 | 128 | Copyright (c) 2015-2018 Benoit Bovy. 129 | 130 | Licensed under the terms of the MIT License 131 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | PAPER = 8 | BUILDDIR = _build 9 | 10 | # User-friendly check for sphinx-build 11 | ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1) 12 | $(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/) 13 | endif 14 | 15 | # Internal variables. 16 | PAPEROPT_a4 = -D latex_paper_size=a4 17 | PAPEROPT_letter = -D latex_paper_size=letter 18 | ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 19 | # the i18n builder cannot share the environment and doctrees with the others 20 | I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 21 | 22 | .PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext 23 | 24 | help: 25 | @echo "Please use \`make ' where is one of" 26 | @echo " html to make standalone HTML files" 27 | @echo " dirhtml to make HTML files named index.html in directories" 28 | @echo " singlehtml to make a single large HTML file" 29 | @echo " pickle to make pickle files" 30 | @echo " json to make JSON files" 31 | @echo " htmlhelp to make HTML files and a HTML help project" 32 | @echo " qthelp to make HTML files and a qthelp project" 33 | @echo " devhelp to make HTML files and a Devhelp project" 34 | @echo " epub to make an epub" 35 | @echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter" 36 | @echo " latexpdf to make LaTeX files and run them through pdflatex" 37 | @echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx" 38 | @echo " text to make text files" 39 | @echo " man to make manual pages" 40 | @echo " texinfo to make Texinfo files" 41 | @echo " info to make Texinfo files and run them through makeinfo" 42 | @echo " gettext to make PO message catalogs" 43 | @echo " changes to make an overview of all changed/added/deprecated items" 44 | @echo " xml to make Docutils-native XML files" 45 | @echo " pseudoxml to make pseudoxml-XML files for display purposes" 46 | @echo " linkcheck to check all external links for integrity" 47 | @echo " doctest to run all doctests embedded in the documentation (if enabled)" 48 | 49 | clean: 50 | rm -rf $(BUILDDIR)/* 51 | 52 | html: 53 | $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html 54 | @echo 55 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/html." 56 | 57 | dirhtml: 58 | $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml 59 | @echo 60 | @echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml." 61 | 62 | singlehtml: 63 | $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml 64 | @echo 65 | @echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml." 66 | 67 | pickle: 68 | $(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle 69 | @echo 70 | @echo "Build finished; now you can process the pickle files." 71 | 72 | json: 73 | $(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json 74 | @echo 75 | @echo "Build finished; now you can process the JSON files." 76 | 77 | htmlhelp: 78 | $(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp 79 | @echo 80 | @echo "Build finished; now you can run HTML Help Workshop with the" \ 81 | ".hhp project file in $(BUILDDIR)/htmlhelp." 82 | 83 | qthelp: 84 | $(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp 85 | @echo 86 | @echo "Build finished; now you can run "qcollectiongenerator" with the" \ 87 | ".qhcp project file in $(BUILDDIR)/qthelp, like this:" 88 | @echo "# qcollectiongenerator $(BUILDDIR)/qthelp/xlrenderer.qhcp" 89 | @echo "To view the help file:" 90 | @echo "# assistant -collectionFile $(BUILDDIR)/qthelp/xlrenderer.qhc" 91 | 92 | devhelp: 93 | $(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp 94 | @echo 95 | @echo "Build finished." 96 | @echo "To view the help file:" 97 | @echo "# mkdir -p $$HOME/.local/share/devhelp/xlrenderer" 98 | @echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/xlrenderer" 99 | @echo "# devhelp" 100 | 101 | epub: 102 | $(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub 103 | @echo 104 | @echo "Build finished. The epub file is in $(BUILDDIR)/epub." 105 | 106 | latex: 107 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 108 | @echo 109 | @echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex." 110 | @echo "Run \`make' in that directory to run these through (pdf)latex" \ 111 | "(use \`make latexpdf' here to do that automatically)." 112 | 113 | latexpdf: 114 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 115 | @echo "Running LaTeX files through pdflatex..." 116 | $(MAKE) -C $(BUILDDIR)/latex all-pdf 117 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 118 | 119 | latexpdfja: 120 | $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex 121 | @echo "Running LaTeX files through platex and dvipdfmx..." 122 | $(MAKE) -C $(BUILDDIR)/latex all-pdf-ja 123 | @echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex." 124 | 125 | text: 126 | $(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text 127 | @echo 128 | @echo "Build finished. The text files are in $(BUILDDIR)/text." 129 | 130 | man: 131 | $(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man 132 | @echo 133 | @echo "Build finished. The manual pages are in $(BUILDDIR)/man." 134 | 135 | texinfo: 136 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 137 | @echo 138 | @echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo." 139 | @echo "Run \`make' in that directory to run these through makeinfo" \ 140 | "(use \`make info' here to do that automatically)." 141 | 142 | info: 143 | $(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo 144 | @echo "Running Texinfo files through makeinfo..." 145 | make -C $(BUILDDIR)/texinfo info 146 | @echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo." 147 | 148 | gettext: 149 | $(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale 150 | @echo 151 | @echo "Build finished. The message catalogs are in $(BUILDDIR)/locale." 152 | 153 | changes: 154 | $(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes 155 | @echo 156 | @echo "The overview file is in $(BUILDDIR)/changes." 157 | 158 | linkcheck: 159 | $(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck 160 | @echo 161 | @echo "Link check complete; look for any errors in the above output " \ 162 | "or in $(BUILDDIR)/linkcheck/output.txt." 163 | 164 | doctest: 165 | $(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest 166 | @echo "Testing of doctests in the sources finished, look at the " \ 167 | "results in $(BUILDDIR)/doctest/output.txt." 168 | 169 | xml: 170 | $(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml 171 | @echo 172 | @echo "Build finished. The XML files are in $(BUILDDIR)/xml." 173 | 174 | pseudoxml: 175 | $(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml 176 | @echo 177 | @echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml." 178 | -------------------------------------------------------------------------------- /docs/authors.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../AUTHORS.rst 2 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # 4 | # xlrenderer documentation build configuration file, created by 5 | # sphinx-quickstart on Tue Jul 9 22:26:36 2013. 6 | # 7 | # This file is execfile()d with the current directory set to its 8 | # containing dir. 9 | # 10 | # Note that not all possible configuration values are present in this 11 | # autogenerated file. 12 | # 13 | # All configuration values have a default; values that are commented out 14 | # serve to show the default. 15 | 16 | import sys 17 | import os 18 | 19 | # If extensions (or modules to document with autodoc) are in another 20 | # directory, add these directories to sys.path here. If the directory is 21 | # relative to the documentation root, use os.path.abspath to make it 22 | # absolute, like shown here. 23 | #sys.path.insert(0, os.path.abspath('.')) 24 | 25 | # Get the project root dir, which is the parent dir of this 26 | cwd = os.getcwd() 27 | project_root = os.path.dirname(cwd) 28 | 29 | # Insert the project root dir as the first element in the PYTHONPATH. 30 | # This lets us ensure that the source package is imported, and that its 31 | # version is used. 32 | sys.path.insert(0, project_root) 33 | 34 | import xlrenderer 35 | 36 | # -- General configuration --------------------------------------------- 37 | 38 | # If your documentation needs a minimal Sphinx version, state it here. 39 | #needs_sphinx = '1.0' 40 | 41 | # Add any Sphinx extension module names here, as strings. They can be 42 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. 43 | extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode'] 44 | 45 | # Add any paths that contain templates here, relative to this directory. 46 | templates_path = ['_templates'] 47 | 48 | # The suffix of source filenames. 49 | source_suffix = '.rst' 50 | 51 | # The encoding of source files. 52 | #source_encoding = 'utf-8-sig' 53 | 54 | # The master toctree document. 55 | master_doc = 'index' 56 | 57 | # General information about the project. 58 | project = u'xlrenderer' 59 | copyright = u'2015, Benoit Bovy' 60 | 61 | # The version info for the project you're documenting, acts as replacement 62 | # for |version| and |release|, also used in various other places throughout 63 | # the built documents. 64 | # 65 | # The short X.Y version. 66 | version = xlrenderer.__version__ 67 | # The full version, including alpha/beta/rc tags. 68 | release = xlrenderer.__version__ 69 | 70 | # The language for content autogenerated by Sphinx. Refer to documentation 71 | # for a list of supported languages. 72 | #language = None 73 | 74 | # There are two options for replacing |today|: either, you set today to 75 | # some non-false value, then it is used: 76 | #today = '' 77 | # Else, today_fmt is used as the format for a strftime call. 78 | #today_fmt = '%B %d, %Y' 79 | 80 | # List of patterns, relative to source directory, that match files and 81 | # directories to ignore when looking for source files. 82 | exclude_patterns = ['_build'] 83 | 84 | # The reST default role (used for this markup: `text`) to use for all 85 | # documents. 86 | #default_role = None 87 | 88 | # If true, '()' will be appended to :func: etc. cross-reference text. 89 | #add_function_parentheses = True 90 | 91 | # If true, the current module name will be prepended to all description 92 | # unit titles (such as .. function::). 93 | #add_module_names = True 94 | 95 | # If true, sectionauthor and moduleauthor directives will be shown in the 96 | # output. They are ignored by default. 97 | #show_authors = False 98 | 99 | # The name of the Pygments (syntax highlighting) style to use. 100 | pygments_style = 'sphinx' 101 | 102 | # A list of ignored prefixes for module index sorting. 103 | #modindex_common_prefix = [] 104 | 105 | # If true, keep warnings as "system message" paragraphs in the built 106 | # documents. 107 | #keep_warnings = False 108 | 109 | 110 | # -- Options for HTML output ------------------------------------------- 111 | 112 | # The theme to use for HTML and HTML Help pages. See the documentation for 113 | # a list of builtin themes. 114 | html_theme = 'default' 115 | 116 | # Theme options are theme-specific and customize the look and feel of a 117 | # theme further. For a list of options available for each theme, see the 118 | # documentation. 119 | #html_theme_options = {} 120 | 121 | # Add any paths that contain custom themes here, relative to this directory. 122 | #html_theme_path = [] 123 | 124 | # The name for this set of Sphinx documents. If None, it defaults to 125 | # " v documentation". 126 | #html_title = None 127 | 128 | # A shorter title for the navigation bar. Default is the same as 129 | # html_title. 130 | #html_short_title = None 131 | 132 | # The name of an image file (relative to this directory) to place at the 133 | # top of the sidebar. 134 | #html_logo = None 135 | 136 | # The name of an image file (within the static path) to use as favicon 137 | # of the docs. This file should be a Windows icon file (.ico) being 138 | # 16x16 or 32x32 pixels large. 139 | #html_favicon = None 140 | 141 | # Add any paths that contain custom static files (such as style sheets) 142 | # here, relative to this directory. They are copied after the builtin 143 | # static files, so a file named "default.css" will overwrite the builtin 144 | # "default.css". 145 | html_static_path = ['_static'] 146 | 147 | # If not '', a 'Last updated on:' timestamp is inserted at every page 148 | # bottom, using the given strftime format. 149 | #html_last_updated_fmt = '%b %d, %Y' 150 | 151 | # If true, SmartyPants will be used to convert quotes and dashes to 152 | # typographically correct entities. 153 | #html_use_smartypants = True 154 | 155 | # Custom sidebar templates, maps document names to template names. 156 | #html_sidebars = {} 157 | 158 | # Additional templates that should be rendered to pages, maps page names 159 | # to template names. 160 | #html_additional_pages = {} 161 | 162 | # If false, no module index is generated. 163 | #html_domain_indices = True 164 | 165 | # If false, no index is generated. 166 | #html_use_index = True 167 | 168 | # If true, the index is split into individual pages for each letter. 169 | #html_split_index = False 170 | 171 | # If true, links to the reST sources are added to the pages. 172 | #html_show_sourcelink = True 173 | 174 | # If true, "Created using Sphinx" is shown in the HTML footer. 175 | # Default is True. 176 | #html_show_sphinx = True 177 | 178 | # If true, "(C) Copyright ..." is shown in the HTML footer. 179 | # Default is True. 180 | #html_show_copyright = True 181 | 182 | # If true, an OpenSearch description file will be output, and all pages 183 | # will contain a tag referring to it. The value of this option 184 | # must be the base URL from which the finished HTML is served. 185 | #html_use_opensearch = '' 186 | 187 | # This is the file name suffix for HTML files (e.g. ".xhtml"). 188 | #html_file_suffix = None 189 | 190 | # Output file base name for HTML help builder. 191 | htmlhelp_basename = 'xlrendererdoc' 192 | 193 | 194 | # -- Options for LaTeX output ------------------------------------------ 195 | 196 | latex_elements = { 197 | # The paper size ('letterpaper' or 'a4paper'). 198 | #'papersize': 'letterpaper', 199 | 200 | # The font size ('10pt', '11pt' or '12pt'). 201 | #'pointsize': '10pt', 202 | 203 | # Additional stuff for the LaTeX preamble. 204 | #'preamble': '', 205 | } 206 | 207 | # Grouping the document tree into LaTeX files. List of tuples 208 | # (source start file, target name, title, author, documentclass 209 | # [howto/manual]). 210 | latex_documents = [ 211 | ('index', 'xlrenderer.tex', 212 | u'xlrenderer Documentation', 213 | u'Benoit Bovy', 'manual'), 214 | ] 215 | 216 | # The name of an image file (relative to this directory) to place at 217 | # the top of the title page. 218 | #latex_logo = None 219 | 220 | # For "manual" documents, if this is true, then toplevel headings 221 | # are parts, not chapters. 222 | #latex_use_parts = False 223 | 224 | # If true, show page references after internal links. 225 | #latex_show_pagerefs = False 226 | 227 | # If true, show URL addresses after external links. 228 | #latex_show_urls = False 229 | 230 | # Documents to append as an appendix to all manuals. 231 | #latex_appendices = [] 232 | 233 | # If false, no module index is generated. 234 | #latex_domain_indices = True 235 | 236 | 237 | # -- Options for manual page output ------------------------------------ 238 | 239 | # One entry per manual page. List of tuples 240 | # (source start file, name, description, authors, manual section). 241 | man_pages = [ 242 | ('index', 'xlrenderer', 243 | u'xlrenderer Documentation', 244 | [u'Benoit Bovy'], 1) 245 | ] 246 | 247 | # If true, show URL addresses after external links. 248 | #man_show_urls = False 249 | 250 | 251 | # -- Options for Texinfo output ---------------------------------------- 252 | 253 | # Grouping the document tree into Texinfo files. List of tuples 254 | # (source start file, target name, title, author, 255 | # dir menu entry, description, category) 256 | texinfo_documents = [ 257 | ('index', 'xlrenderer', 258 | u'xlrenderer Documentation', 259 | u'Benoit Bovy', 260 | 'xlrenderer', 261 | 'One line description of project.', 262 | 'Miscellaneous'), 263 | ] 264 | 265 | # Documents to append as an appendix to all manuals. 266 | #texinfo_appendices = [] 267 | 268 | # If false, no module index is generated. 269 | #texinfo_domain_indices = True 270 | 271 | # How to display URL addresses: 'footnote', 'no', or 'inline'. 272 | #texinfo_show_urls = 'footnote' 273 | 274 | # If true, do not generate a @detailmenu in the "Top" node's menu. 275 | #texinfo_no_detailmenu = False 276 | -------------------------------------------------------------------------------- /docs/contributing.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../CONTRIBUTING.rst 2 | -------------------------------------------------------------------------------- /docs/history.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../HISTORY.rst 2 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. xlrenderer documentation master file, created by 2 | sphinx-quickstart on Tue Jul 9 22:26:36 2013. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | Welcome to xlrenderer's documentation! 7 | ====================================== 8 | 9 | Contents: 10 | 11 | .. toctree:: 12 | :maxdepth: 2 13 | 14 | readme 15 | installation 16 | usage 17 | contributing 18 | authors 19 | history 20 | 21 | Indices and tables 22 | ================== 23 | 24 | * :ref:`genindex` 25 | * :ref:`modindex` 26 | * :ref:`search` 27 | 28 | -------------------------------------------------------------------------------- /docs/installation.rst: -------------------------------------------------------------------------------- 1 | ============ 2 | Installation 3 | ============ 4 | 5 | At the command line:: 6 | 7 | $ easy_install xlrenderer 8 | 9 | Or, if you have virtualenvwrapper installed:: 10 | 11 | $ mkvirtualenv xlrenderer 12 | $ pip install xlrenderer 13 | -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | REM Command file for Sphinx documentation 4 | 5 | if "%SPHINXBUILD%" == "" ( 6 | set SPHINXBUILD=sphinx-build 7 | ) 8 | set BUILDDIR=_build 9 | set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% . 10 | set I18NSPHINXOPTS=%SPHINXOPTS% . 11 | if NOT "%PAPER%" == "" ( 12 | set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS% 13 | set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS% 14 | ) 15 | 16 | if "%1" == "" goto help 17 | 18 | if "%1" == "help" ( 19 | :help 20 | echo.Please use `make ^` where ^ is one of 21 | echo. html to make standalone HTML files 22 | echo. dirhtml to make HTML files named index.html in directories 23 | echo. singlehtml to make a single large HTML file 24 | echo. pickle to make pickle files 25 | echo. json to make JSON files 26 | echo. htmlhelp to make HTML files and a HTML help project 27 | echo. qthelp to make HTML files and a qthelp project 28 | echo. devhelp to make HTML files and a Devhelp project 29 | echo. epub to make an epub 30 | echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter 31 | echo. text to make text files 32 | echo. man to make manual pages 33 | echo. texinfo to make Texinfo files 34 | echo. gettext to make PO message catalogs 35 | echo. changes to make an overview over all changed/added/deprecated items 36 | echo. xml to make Docutils-native XML files 37 | echo. pseudoxml to make pseudoxml-XML files for display purposes 38 | echo. linkcheck to check all external links for integrity 39 | echo. doctest to run all doctests embedded in the documentation if enabled 40 | goto end 41 | ) 42 | 43 | if "%1" == "clean" ( 44 | for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i 45 | del /q /s %BUILDDIR%\* 46 | goto end 47 | ) 48 | 49 | 50 | %SPHINXBUILD% 2> nul 51 | if errorlevel 9009 ( 52 | echo. 53 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 54 | echo.installed, then set the SPHINXBUILD environment variable to point 55 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 56 | echo.may add the Sphinx directory to PATH. 57 | echo. 58 | echo.If you don't have Sphinx installed, grab it from 59 | echo.http://sphinx-doc.org/ 60 | exit /b 1 61 | ) 62 | 63 | if "%1" == "html" ( 64 | %SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html 65 | if errorlevel 1 exit /b 1 66 | echo. 67 | echo.Build finished. The HTML pages are in %BUILDDIR%/html. 68 | goto end 69 | ) 70 | 71 | if "%1" == "dirhtml" ( 72 | %SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml 73 | if errorlevel 1 exit /b 1 74 | echo. 75 | echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml. 76 | goto end 77 | ) 78 | 79 | if "%1" == "singlehtml" ( 80 | %SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml 81 | if errorlevel 1 exit /b 1 82 | echo. 83 | echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml. 84 | goto end 85 | ) 86 | 87 | if "%1" == "pickle" ( 88 | %SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle 89 | if errorlevel 1 exit /b 1 90 | echo. 91 | echo.Build finished; now you can process the pickle files. 92 | goto end 93 | ) 94 | 95 | if "%1" == "json" ( 96 | %SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json 97 | if errorlevel 1 exit /b 1 98 | echo. 99 | echo.Build finished; now you can process the JSON files. 100 | goto end 101 | ) 102 | 103 | if "%1" == "htmlhelp" ( 104 | %SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp 105 | if errorlevel 1 exit /b 1 106 | echo. 107 | echo.Build finished; now you can run HTML Help Workshop with the ^ 108 | .hhp project file in %BUILDDIR%/htmlhelp. 109 | goto end 110 | ) 111 | 112 | if "%1" == "qthelp" ( 113 | %SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp 114 | if errorlevel 1 exit /b 1 115 | echo. 116 | echo.Build finished; now you can run "qcollectiongenerator" with the ^ 117 | .qhcp project file in %BUILDDIR%/qthelp, like this: 118 | echo.^> qcollectiongenerator %BUILDDIR%\qthelp\xlrenderer.qhcp 119 | echo.To view the help file: 120 | echo.^> assistant -collectionFile %BUILDDIR%\qthelp\xlrenderer.ghc 121 | goto end 122 | ) 123 | 124 | if "%1" == "devhelp" ( 125 | %SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp 126 | if errorlevel 1 exit /b 1 127 | echo. 128 | echo.Build finished. 129 | goto end 130 | ) 131 | 132 | if "%1" == "epub" ( 133 | %SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub 134 | if errorlevel 1 exit /b 1 135 | echo. 136 | echo.Build finished. The epub file is in %BUILDDIR%/epub. 137 | goto end 138 | ) 139 | 140 | if "%1" == "latex" ( 141 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 142 | if errorlevel 1 exit /b 1 143 | echo. 144 | echo.Build finished; the LaTeX files are in %BUILDDIR%/latex. 145 | goto end 146 | ) 147 | 148 | if "%1" == "latexpdf" ( 149 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 150 | cd %BUILDDIR%/latex 151 | make all-pdf 152 | cd %BUILDDIR%/.. 153 | echo. 154 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 155 | goto end 156 | ) 157 | 158 | if "%1" == "latexpdfja" ( 159 | %SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex 160 | cd %BUILDDIR%/latex 161 | make all-pdf-ja 162 | cd %BUILDDIR%/.. 163 | echo. 164 | echo.Build finished; the PDF files are in %BUILDDIR%/latex. 165 | goto end 166 | ) 167 | 168 | if "%1" == "text" ( 169 | %SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text 170 | if errorlevel 1 exit /b 1 171 | echo. 172 | echo.Build finished. The text files are in %BUILDDIR%/text. 173 | goto end 174 | ) 175 | 176 | if "%1" == "man" ( 177 | %SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man 178 | if errorlevel 1 exit /b 1 179 | echo. 180 | echo.Build finished. The manual pages are in %BUILDDIR%/man. 181 | goto end 182 | ) 183 | 184 | if "%1" == "texinfo" ( 185 | %SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo 186 | if errorlevel 1 exit /b 1 187 | echo. 188 | echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo. 189 | goto end 190 | ) 191 | 192 | if "%1" == "gettext" ( 193 | %SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale 194 | if errorlevel 1 exit /b 1 195 | echo. 196 | echo.Build finished. The message catalogs are in %BUILDDIR%/locale. 197 | goto end 198 | ) 199 | 200 | if "%1" == "changes" ( 201 | %SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes 202 | if errorlevel 1 exit /b 1 203 | echo. 204 | echo.The overview file is in %BUILDDIR%/changes. 205 | goto end 206 | ) 207 | 208 | if "%1" == "linkcheck" ( 209 | %SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck 210 | if errorlevel 1 exit /b 1 211 | echo. 212 | echo.Link check complete; look for any errors in the above output ^ 213 | or in %BUILDDIR%/linkcheck/output.txt. 214 | goto end 215 | ) 216 | 217 | if "%1" == "doctest" ( 218 | %SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest 219 | if errorlevel 1 exit /b 1 220 | echo. 221 | echo.Testing of doctests in the sources finished, look at the ^ 222 | results in %BUILDDIR%/doctest/output.txt. 223 | goto end 224 | ) 225 | 226 | if "%1" == "xml" ( 227 | %SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml 228 | if errorlevel 1 exit /b 1 229 | echo. 230 | echo.Build finished. The XML files are in %BUILDDIR%/xml. 231 | goto end 232 | ) 233 | 234 | if "%1" == "pseudoxml" ( 235 | %SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml 236 | if errorlevel 1 exit /b 1 237 | echo. 238 | echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml. 239 | goto end 240 | ) 241 | 242 | :end 243 | -------------------------------------------------------------------------------- /docs/readme.rst: -------------------------------------------------------------------------------- 1 | .. include:: ../README.rst 2 | -------------------------------------------------------------------------------- /docs/usage.rst: -------------------------------------------------------------------------------- 1 | ======== 2 | Usage 3 | ======== 4 | 5 | To use xlrenderer in a project:: 6 | 7 | import xlrenderer 8 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | sqlalchemy 2 | pandas 3 | xlwings 4 | pyyaml 5 | jinja2 6 | 7 | -------------------------------------------------------------------------------- /scripts/render_access2xls: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # encoding: utf-8 3 | 4 | """ 5 | Render an Excel template from an Access Database using a specification file. 6 | """ 7 | 8 | import logging 9 | import sys 10 | import argparse 11 | import os 12 | import signal 13 | 14 | 15 | WINDOWS = sys.platform == 'win32' 16 | 17 | # use with pynsist (needed to properly import pywin modules and xlwings) 18 | try: 19 | script_dir, script = os.path.split(__file__) 20 | except NameError: 21 | script_dir = os.path.dirname(sys.argv[0]) 22 | script_dir_path = os.path.abspath(script_dir) 23 | pkgs_path = os.path.join(script_dir_path, 'pkgs') 24 | if os.path.isdir(pkgs_path) and WINDOWS: 25 | sys.path.insert(0, pkgs_path) 26 | sys.path.insert(0, os.path.join(pkgs_path, 'win32')) 27 | sys.path.insert(0, os.path.join(pkgs_path, 'win32', 'lib')) 28 | os.environ['PATH'] = ';'.join( 29 | [os.path.join(pkgs_path, 'pywin32_system32'), os.environ['PATH']] 30 | ) 31 | os.environ['PATH'] = ';'.join( 32 | [os.path.join(pkgs_path, 'xlwings_dlls'), os.environ['PATH']] 33 | ) 34 | 35 | 36 | import xlrenderer 37 | from xlrenderer.utils import create_access_engine 38 | 39 | 40 | logger = logging.getLogger(__name__) 41 | 42 | _LOG_LEVEL_STRINGS = ['critical', 'error', 'warning', 'info', 'debug'] 43 | 44 | 45 | def _log_level_string_to_int(log_level_string): 46 | """ 47 | Allow providing either number or string for log level argument. 48 | """ 49 | if not log_level_string in _LOG_LEVEL_STRINGS: 50 | message = 'invalid choice: {0} (choose from {1})'.format( 51 | log_level_string, _LOG_LEVEL_STRINGS 52 | ) 53 | raise argparse.ArgumentTypeError(message) 54 | 55 | log_level_int = getattr(logging, log_level_string.upper(), logging.INFO) 56 | # check the logging log_level_choices have not changed 57 | # from our expected values 58 | assert isinstance(log_level_int, int) 59 | 60 | return log_level_int 61 | 62 | 63 | def parse_command_line(argv): 64 | """ 65 | Parse command line argument. See -h option. 66 | """ 67 | formatter_class = argparse.RawDescriptionHelpFormatter 68 | parser = argparse.ArgumentParser( 69 | formatter_class=formatter_class, 70 | description="Render an Excel template from an Access Database using " 71 | "a specification file." 72 | ) 73 | parser.add_argument( 74 | "-a", "--accdb", required=True, dest="access_dbfile", 75 | help="name or path to the Access database file (*.mdb, *.accdb)" 76 | ) 77 | parser.add_argument( 78 | "-t", "--template", required=True, dest="wkb_template", 79 | help="name or path to the Excel template file (*.xls, *.xlsx)" 80 | ) 81 | parser.add_argument( 82 | "-s", "--specfile", required=True, dest="spec_filename", 83 | help="name or path to the specification file (YAML format)" 84 | ) 85 | parser.add_argument( 86 | "-o", "--outputdir", default=os.curdir, dest="output_dir", 87 | help="output directory (will be created if doesn't exist)" 88 | ) 89 | parser.add_argument( 90 | "-l", "--log-level", default="info", dest="log_level", 91 | type=_log_level_string_to_int, nargs='?', 92 | help="Set the logging output level. {0}".format(_LOG_LEVEL_STRINGS) 93 | ) 94 | args = parser.parse_args(argv[1:]) 95 | 96 | return args 97 | 98 | 99 | def sig_break_handler(sig, frame): 100 | raise KeyboardInterrupt() 101 | 102 | if hasattr(signal, "SIGBREAK"): 103 | # Handle Ctrl-Break e.g. under Windows 104 | signal.signal(signal.SIGBREAK, sig_break_handler) 105 | 106 | 107 | def main(): 108 | args = parse_command_line(sys.argv) 109 | logging.basicConfig(stream=sys.stderr, level=args.log_level, 110 | format='%(name)s (%(levelname)s): %(message)s') 111 | try: 112 | access_engine = create_access_engine(args.access_dbfile) 113 | jenv = xlrenderer.utils.jinja_custom_env 114 | r = xlrenderer.ExcelTemplateRenderer(access_engine, 115 | args.wkb_template, 116 | args.spec_filename, 117 | args.output_dir, 118 | jinja_env=jenv) 119 | r.render() 120 | except (KeyboardInterrupt, SystemExit): 121 | r.wkb.close() # close the current open workbook 122 | logger.info("Excel workbook closed") 123 | logger.warning('Program interrupted!') 124 | finally: 125 | logging.shutdown() 126 | 127 | 128 | if __name__ == "__main__": 129 | sys.exit(main()) 130 | -------------------------------------------------------------------------------- /scripts/render_access2xls_gui: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | """ 5 | Access DB - Excel Template Renderer with a basic GUI (Tkinter). 6 | """ 7 | 8 | from collections import OrderedDict 9 | from queue import Queue, Empty 10 | import threading 11 | import signal 12 | import logging 13 | import sys 14 | import os 15 | import subprocess 16 | 17 | import tkinter as tk 18 | from tkinter import ttk 19 | from tkinter.filedialog import askopenfilename, askdirectory 20 | from tkinter.scrolledtext import ScrolledText 21 | from tkinter.messagebox import showerror, showinfo, askyesno 22 | 23 | 24 | PROG_NAME = "Access2xls" 25 | PROG_VERSION = "0.1.0" 26 | PROG_YEAR = "2015" 27 | PROG_AUTHORS = "Benoit Bovy" 28 | PROG_LICENSE = "MIT" 29 | 30 | LOG_LEVELS = OrderedDict([ 31 | ('all', 0), 32 | ('debug', 10), 33 | ('info', 20), 34 | ('warning', 30), 35 | ('error', 40), 36 | ('critical', 50) 37 | ]) 38 | 39 | WINDOWS = sys.platform == 'win32' 40 | 41 | 42 | class MainFrame(ttk.Frame): 43 | def __init__(self, parent=None): 44 | ttk.Frame.__init__(self, parent, padding=(15, 15, 12, 12)) 45 | self.parent = parent 46 | self.init_ui() 47 | 48 | def init_ui(self): 49 | 50 | self.parent.title("Access DB - Excel Template Renderer") 51 | self.parent.lift() 52 | self.parent.columnconfigure(0, weight=1) 53 | self.parent.rowconfigure(0, weight=1) 54 | 55 | self.style = ttk.Style() 56 | #self.style.theme_use("clam") 57 | 58 | self.grid(row=0, column=0, sticky=(tk.N, tk.S, tk.E, tk.W)) 59 | self.rowconfigure(3, weight=1) 60 | self.columnconfigure(0, weight=1) 61 | 62 | # INPUT FRAME: input files / directories 63 | self.input_frame = ttk.Frame(self) 64 | self.input_frame.grid(column=0, row=0, sticky=(tk.N, tk.E, tk.W)) 65 | self.input_frame.columnconfigure(1, weight=5) 66 | self.input_frame.columnconfigure(2, weight=1) 67 | self.input_vars = {} 68 | self.input_labels = {} 69 | self.input_entries = {} 70 | self.input_buttons = {} 71 | self.create_input_ui( 72 | 'access_dbfile', 0, "Access DB File: ", 73 | title="Choose an Access database file", 74 | filetypes=(("Access DB files", (".accdb", ".mdb")),) 75 | ) 76 | self.create_input_ui( 77 | 'wkb_template', 1, "XLS Template: ", 78 | title="Choose an Excel file as template", 79 | filetypes=(("Excel files", (".xls", ".xlsx")),) 80 | ) 81 | self.create_input_ui( 82 | 'spec_filename', 2, "Specification File: ", 83 | title="Choose a specification file (YAML format)", 84 | filetypes=(("YAML files", (".yml", ".yaml")),) 85 | ) 86 | self.create_input_ui( 87 | 'output_dir', 3, "Output Directory: ", 88 | dialog=askdirectory, 89 | title="Choose an output directory", 90 | mustexist=False, 91 | ) 92 | 93 | # RUN FRAME: log-level select, progress bar and render/stop buttons 94 | self.run_frame = ttk.Frame(self) 95 | self.run_frame.grid(column=0, row=1, sticky=(tk.W, tk.E), pady=10) 96 | self.run_frame.columnconfigure(2, weight=1) 97 | 98 | self.loglevel_var = tk.StringVar() 99 | self.log_level_label = ttk.Label(self.run_frame, text="Log Level: ") 100 | self.loglevel_option = ttk.OptionMenu( 101 | self.run_frame, self.loglevel_var, *LOG_LEVELS 102 | ) 103 | self.loglevel_var.set('info') 104 | 105 | self.progress_bar = ttk.Progressbar( 106 | self.run_frame, orient="horizontal", 107 | length=200, mode="indeterminate" 108 | ) 109 | 110 | self.render_button = ttk.Button( 111 | self.run_frame, text="Render!", width=10, 112 | command=self.action_start_render 113 | ) 114 | self.stop_button = ttk.Button( 115 | self.run_frame, text="Stop!", width=10, state='disabled', 116 | command=self.action_stop_render 117 | ) 118 | 119 | self.log_level_label.grid(row=0, column=0, sticky=tk.W) 120 | self.loglevel_option.grid(row=0, column=1, sticky=tk.W) 121 | self.progress_bar.grid(row=0, column=2, sticky=(tk.W, tk.E), padx=20) 122 | self.render_button.grid(row=0, column=3, sticky=(tk.W, tk.E)) 123 | self.stop_button.grid(row=0, column=4, sticky=tk.E) 124 | 125 | # CONSOLE: scrolled-text 126 | self.console = ScrolledText(self, state='disabled') 127 | self.console.configure(font='TkFixedFont') 128 | self.console.grid(column=0, row=3, sticky=(tk.N, tk.S, tk.E, tk.W), 129 | pady=10) 130 | 131 | # MENU BAR 132 | self.menubar = tk.Menu(self.parent) 133 | self.parent.config(menu=self.menubar) 134 | 135 | self.filemenu = tk.Menu(self.menubar) 136 | self.filemenu.add_command(label="Quit", command=self.action_quit) 137 | self.menubar.add_cascade(label="File", menu=self.filemenu) 138 | 139 | self.helpmenu = tk.Menu(self.menubar) 140 | self.helpmenu.add_command(label="About", 141 | command=self.action_show_about) 142 | self.menubar.add_cascade(label="Help", menu=self.helpmenu) 143 | 144 | def create_input_ui(self, name, row, label_text, 145 | dialog=askopenfilename, **kwargs): 146 | """Create an input (label + entry + filedialog button)""" 147 | self.input_vars[name] = tk.StringVar() 148 | self.input_labels[name] = ttk.Label(self.input_frame, text=label_text) 149 | self.input_entries[name] = ttk.Entry( 150 | self.input_frame, textvariable=self.input_vars[name] 151 | ) 152 | self.input_buttons[name] = ttk.Button( 153 | self.input_frame, text="...", width=4, 154 | command=lambda: self.action_set_input(name, dialog, **kwargs) 155 | ) 156 | self.input_labels[name].grid(row=row, column=0, sticky=tk.W) 157 | self.input_entries[name].grid(row=row, column=1, sticky=(tk.E, tk.W)) 158 | self.input_buttons[name].grid(row=row, column=2, sticky=(tk.E, tk.W)) 159 | 160 | def action_quit(self): 161 | if askyesno("Quit", "Quit? Ensure no render task is running"): 162 | self.parent.quit() 163 | return 164 | 165 | def action_show_about(self): 166 | about_msg = "{}\nVersion: {}\nCopyright {}, {}\nLicense: {}".format( 167 | PROG_NAME, PROG_VERSION, PROG_AUTHORS, PROG_YEAR, PROG_LICENSE 168 | ) 169 | showinfo(title="About {}".format(PROG_NAME), message=about_msg) 170 | return 171 | 172 | def action_set_input(self, name, dialog, **kwargs): 173 | """Open a dialog and get the choosen file or directory.""" 174 | if kwargs.get('filetypes') is not None: 175 | kwargs['filetypes'] = list(kwargs['filetypes']) 176 | kwargs['filetypes'] += [("All files", ".*")] 177 | input_name = dialog(parent=self.parent, **kwargs) 178 | self.input_vars[name].set(input_name) 179 | 180 | def action_start_render(self): 181 | for k, v in self.input_vars.items(): 182 | if v.get(): 183 | continue 184 | showerror(title="Input missing", 185 | message="Please specify {}".format( 186 | self.input_labels[k]['text'] 187 | )) 188 | return 189 | 190 | self.clear_console() 191 | self.render_button.config(state='disabled') 192 | self.progress_bar.start() 193 | 194 | # set the command line 195 | cmd = [ 196 | sys.executable, 197 | os.path.join(os.path.dirname(sys.argv[0]), 'render_access2xls'), 198 | '--accdb', '{}'.format(self.input_vars['access_dbfile'].get()), 199 | '--template', '{}'.format(self.input_vars['wkb_template'].get()), 200 | '--specfile', '{}'.format(self.input_vars['spec_filename'].get()), 201 | '--outputdir', '{}'.format(self.input_vars['output_dir'].get()), 202 | '--log-level', self.loglevel_var.get(), 203 | ] 204 | 205 | # execute the command 206 | kwargs = {} 207 | if WINDOWS: 208 | kwargs['creationflags'] = subprocess.CREATE_NEW_PROCESS_GROUP 209 | 210 | self.render_task = subprocess.Popen( 211 | cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, **kwargs 212 | ) 213 | 214 | # non-blocking reading of the output of the subprocess 215 | self.render_task_queue = Queue(maxsize=1) 216 | t = threading.Thread(target=self.enqueue_output, 217 | args=(self.render_task.stderr, 218 | self.render_task_queue)) 219 | t.daemon = True # thread dies with the program 220 | t.start() 221 | 222 | # enable stop button and start monitoring the process 223 | self.stop_button.config(state='enabled') 224 | self.parent.after(100, self.monitor_render_task) 225 | 226 | def action_stop_render(self): 227 | if WINDOWS: 228 | os.kill(self.render_task.pid, signal.CTRL_BREAK_EVENT) 229 | else: 230 | self.render_task.terminate() 231 | 232 | self.stop_button.config(state='disabled') 233 | self.render_button.config(state='enabled') 234 | 235 | def monitor_render_task(self): 236 | if self.render_task.poll() is not None: 237 | try: 238 | # get task remaining outputs 239 | line = self.render_task_queue.get_nowait() 240 | self.append_console(line) 241 | self.parent.after(100, self.monitor_render_task) 242 | except Empty: 243 | # consider the task finished 244 | self.progress_bar.stop() 245 | self.render_button.config(state='enabled') 246 | self.stop_button.config(state='disabled') 247 | return 248 | try: 249 | line = self.render_task_queue.get_nowait() 250 | self.append_console(line) 251 | except Empty: 252 | pass 253 | finally: 254 | self.parent.after(100, self.monitor_render_task) 255 | 256 | def enqueue_output(self, out, queue): 257 | for line in iter(out.readline, b''): 258 | queue.put(line) 259 | out.close() 260 | 261 | def clear_console(self): 262 | self.console.configure(state='normal') 263 | self.console.delete('0.0', tk.END) 264 | self.console.configure(state='disabled') 265 | 266 | def append_console(self, msg): 267 | self.console.configure(state='normal') 268 | self.console.insert(tk.END, msg) 269 | self.console.configure(state='disabled') 270 | self.console.yview(tk.END) 271 | 272 | 273 | def main(): 274 | logging.basicConfig(stream=sys.stderr, level=logging.DEBUG, 275 | format='%(name)s (%(levelname)s): %(message)s') 276 | 277 | root = tk.Tk() 278 | root.geometry("800x580+100+100") 279 | root.minsize(400, 400) 280 | app = MainFrame(root) 281 | root.mainloop() 282 | 283 | if __name__ == '__main__': 284 | main() 285 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | 4 | 5 | try: 6 | from setuptools import setup 7 | except ImportError: 8 | from distutils.core import setup 9 | 10 | 11 | with open('README.rst') as readme_file: 12 | readme = readme_file.read() 13 | 14 | with open('HISTORY.rst') as history_file: 15 | history = history_file.read().replace('.. :changelog:', '') 16 | 17 | requirements = [ 18 | # TODO: put package requirements here 19 | ] 20 | 21 | test_requirements = [ 22 | # TODO: put package test requirements here 23 | ] 24 | 25 | setup( 26 | name='xlrenderer', 27 | version='0.1.0', 28 | description="Render excel templates using a database and a specification file", 29 | long_description=readme + '\n\n' + history, 30 | author="Benoit Bovy", 31 | author_email='benbovy@gmail.com', 32 | url='https://github.com/benbovy/xlrenderer', 33 | packages=[ 34 | 'xlrenderer', 35 | ], 36 | package_dir={'xlrenderer': 37 | 'xlrenderer'}, 38 | include_package_data=True, 39 | scripts=[ 40 | 'scripts/render_access2xls', 41 | 'scripts/render_access2xls_gui' 42 | ], 43 | install_requires=requirements, 44 | license="ISCL", 45 | zip_safe=False, 46 | keywords='xlrenderer', 47 | classifiers=[ 48 | 'Development Status :: 2 - Pre-Alpha', 49 | 'Intended Audience :: Developers', 50 | 'License :: OSI Approved :: ISC License (ISCL)', 51 | 'Natural Language :: English', 52 | "Programming Language :: Python :: 2", 53 | 'Programming Language :: Python :: 2.6', 54 | 'Programming Language :: Python :: 2.7', 55 | 'Programming Language :: Python :: 3', 56 | 'Programming Language :: Python :: 3.3', 57 | 'Programming Language :: Python :: 3.4', 58 | ], 59 | test_suite='tests', 60 | tests_require=test_requirements 61 | ) 62 | -------------------------------------------------------------------------------- /xlrenderer/__init__.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | __author__ = 'Benoit Bovy' 4 | __email__ = 'benbovy@gmail.com' 5 | __version__ = '0.1.0' 6 | 7 | 8 | from .xlrenderer import ExcelTemplateRenderer 9 | from .utils import * 10 | -------------------------------------------------------------------------------- /xlrenderer/utils.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | """ 4 | Utils: database connections/engines and jinja2 custom filters. 5 | 6 | """ 7 | 8 | import numpy as np 9 | import sqlalchemy 10 | import pypyodbc 11 | import jinja2 12 | 13 | 14 | __all__ = ["connect_access_db", "create_access_engine", "jinja_custom_env"] 15 | 16 | 17 | def connect_access_db(filename): 18 | """Returns a connection to the Access database.""" 19 | 20 | driver = '{Microsoft Access Driver (*.mdb, *.accdb)}' 21 | con = pypyodbc.connect('Driver={0};Dbq={1};Uid=Admin;Pwd=;' 22 | .format(driver, filename)) 23 | return con 24 | 25 | def create_access_engine(filename): 26 | """ 27 | Creates a new SQLAlchemy engine from an Access 28 | database (.mdb, .accdb). 29 | """ 30 | engine = sqlalchemy.create_engine( 31 | 'mysql+pyodbc://', 32 | creator=lambda: connect_access_db(filename) 33 | ) 34 | return engine 35 | 36 | 37 | def none2empty_filter(val): 38 | """Jinja2 template to convert None value to empty string.""" 39 | if not val is None: 40 | return val 41 | else: 42 | return '' 43 | 44 | def nan2empty_filter(val): 45 | """Jinja2 template to convert 'nan' value to empty string.""" 46 | try: 47 | if np.isnan(val): 48 | return '' 49 | except TypeError: 50 | pass 51 | return val 52 | 53 | jinja_custom_env = jinja2.Environment() 54 | jinja_custom_env.filters['none2empty'] = none2empty_filter 55 | jinja_custom_env.filters['nan2empty'] = nan2empty_filter 56 | -------------------------------------------------------------------------------- /xlrenderer/xlrenderer.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | 3 | """ 4 | 5 | """ 6 | import os 7 | import logging 8 | 9 | import pandas as pd 10 | 11 | from xlwings import Workbook, Sheet, Range 12 | 13 | import yaml 14 | import jinja2 15 | 16 | 17 | logger = logging.getLogger('xlrenderer') 18 | 19 | 20 | class ExcelTemplateRenderer(object): 21 | """ 22 | A class to render an Excel template using data stored 23 | in a database and given a specification file. 24 | 25 | Parameters 26 | ---------- 27 | db_engine : object 28 | the :class:`sqlalchemy.engine.Engine` object used to 29 | connect and query the database. 30 | template_name : str 31 | path to the `xlsx` template file. 32 | spec_filename : str 33 | path to the specification file (yaml format). 34 | output_dirname : str 35 | path to the directory where to store the 36 | generated excel file(s) (i.e., the rendered template(s)). 37 | jinja_env : object or None 38 | allow to provide a :class:`jinja2.Environment` object, 39 | helpful when using custom filters in the specification file 40 | (optional). 41 | 42 | Notes 43 | ----- 44 | The specification file (yaml format) consists of a list 45 | of render blocks. TODO: documentation on the format. 46 | 47 | """ 48 | 49 | def __init__(self, db_engine, template_name, 50 | spec_filename, output_dirname, jinja_env=None): 51 | 52 | self.db_engine = db_engine 53 | self.template_name = template_name 54 | self.spec_filename = spec_filename 55 | self.output_dirname = os.path.abspath(output_dirname) 56 | 57 | if jinja_env is None: 58 | self.jinja_env = jinja2.Environment() 59 | else: 60 | self.jinja_env = jinja_env 61 | 62 | with open(self.spec_filename, 'r', encoding='utf-8') as f: 63 | self.render_blocks = yaml.load(f) 64 | 65 | os.makedirs(self.output_dirname, exist_ok=True) 66 | logger.info("output directory is %s", self.output_dirname) 67 | 68 | def open_template_as_current_wkb(self): 69 | self.wkb = Workbook( 70 | os.path.abspath(self.template_name), 71 | app_visible=False 72 | ) 73 | self.wkb.set_current() 74 | 75 | def save_current_wkb(self, filename): 76 | filepath = os.path.join(self.output_dirname, filename) 77 | self.wkb.save(filepath) 78 | logger.info("created %s", filepath) 79 | 80 | def save_current_wkb_as_pdf(self, filename, worksheet_name): 81 | filepath = os.path.join(self.output_dirname, filename) 82 | try: 83 | ws = Sheet(worksheet_name, wkb=self.wkb) 84 | ws.xl_sheet.ExportAsFixedFormat(0, filepath) 85 | logger.info("created %s", filepath) 86 | except Exception as e: 87 | logger.error("failed to export pdf") 88 | logger.error("detailled error: %s - %s", 89 | e.__class__.__name__, str(e)) 90 | 91 | def close_current_wkb(self): 92 | self.wkb.close() 93 | 94 | def insert_one_series(self, series, cell_specification): 95 | """ 96 | Populate the current workbook given a single 97 | :class=:`pandas.Series` object. 98 | """ 99 | if not len(series): 100 | return 101 | 102 | # contiguous cells 103 | #TODO: (use vertical and horizontal properties of xlwings) 104 | 105 | # non-contiguous user-defined cells 106 | for cs in cell_specification.get('cells', []): 107 | ws = cs.get('worksheet') or Sheet.active(self.wkb).name 108 | content = self.jinja_env.from_string(cs['content']).render(**series) 109 | 110 | logger.debug("insert content '%s' at cell '%s' in sheet '%s'", 111 | content, cs['cell'], ws) 112 | 113 | Range(ws, cs['cell']).value = content 114 | 115 | def insert_one_dataframe(self, df, cell_specification): 116 | """ 117 | Populate the current workbook given a single 118 | :class=:`pandas.DataFrame` object. 119 | """ 120 | if not len(df): 121 | return 122 | 123 | index = cell_specification.get('index', False) 124 | header = cell_specification.get('header', False) 125 | top_left_cell = cell_specification.get('top_left_cell', 'A0') 126 | 127 | logger.debug("insert %d by %d rows/cols dataframe " 128 | "at cell '%s' in sheet '%s'", 129 | len(df), len(df.columns), 130 | str(top_left_cell), Sheet.active(self.wkb).name) 131 | 132 | Range(top_left_cell, index=index, header=header).value = df 133 | 134 | def apply_render_block(self, render_block, query_context=None, 135 | **kwargs): 136 | """ 137 | Apply a single render block in the specification file. 138 | 139 | - `query_context` (mappable or None) is a context used when 140 | rendering the database query with jinja2 (optional). 141 | - **kwargs is used to overwrite any key/value pair 142 | in the render block. 143 | 144 | """ 145 | # override render_block key/val with kwargs 146 | render_block.update(kwargs) 147 | 148 | logger.info("processing render block '%s'", 149 | render_block.get('name', '')) 150 | 151 | # query the DB into a pandas DataFrame 152 | if query_context is None: 153 | query_context = dict() 154 | query_template = self.jinja_env.from_string( 155 | render_block['query'].strip() 156 | ) 157 | query = query_template.render(**query_context) 158 | logger.debug("rendered query: \n'''\n%s\n'''", query) 159 | df = pd.read_sql(query, self.db_engine) 160 | 161 | logger.debug("query returned %d record(s)", len(df)) 162 | 163 | # TODO: calculate extra columns and add it to the DataFrame 164 | 165 | # activate worksheet if provided 166 | ws_name = render_block['cell_specification'].get('worksheet') or None 167 | if ws_name is not None: 168 | ws2reactivate_name = Sheet.active(self.wkb).name 169 | Sheet(ws_name, wkb=self.wkb).activate() 170 | 171 | # apply the render_block, apply recusively included blocks, 172 | # and save the rendered workbook(s) if needed 173 | apply_by_row = render_block.get('apply_by_row', False) 174 | save_as = render_block.get('save_as', None) 175 | 176 | if apply_by_row and save_as is not None: 177 | logger.info("%d file(s) to generate", len(df)) 178 | 179 | if apply_by_row: 180 | for row, pseries in df.iterrows(): 181 | self.insert_one_series( 182 | pseries, render_block['cell_specification'] 183 | ) 184 | 185 | for item in render_block.get('include', []): 186 | if isinstance(item, dict): 187 | block_name = item.pop('render_block') 188 | override_vars = item 189 | else: 190 | block_name = item 191 | override_vars = {} 192 | block = [b for b in self.render_blocks 193 | if b['name'] == block_name][0] 194 | self.apply_render_block(block, 195 | query_context=pseries, 196 | **override_vars) 197 | 198 | if save_as is not None: 199 | tpl = save_as['filename'] 200 | filename = self.jinja_env.from_string(tpl).render(**pseries) 201 | self.save_current_wkb(filename) 202 | 203 | # save to pdf 204 | if save_as.get('export_pdf', False): 205 | filename_pdf = os.path.splitext(filename)[0] + '.pdf' 206 | if ws_name is None: 207 | logger.error( 208 | "(export to pdf) no worksheet specified" 209 | ) 210 | else: 211 | self.save_current_wkb_as_pdf(filename_pdf, ws_name) 212 | 213 | # re-open the template, re-activate the worksheet 214 | self.close_current_wkb() 215 | self.open_template_as_current_wkb() 216 | if ws_name is not None: 217 | Sheet(ws_name, wkb=self.wkb).activate() 218 | 219 | else: 220 | self.insert_one_dataframe( 221 | df, render_block['cell_specification'] 222 | ) 223 | # TODO: include and save_as in this case 224 | 225 | # re-activate former worksheet if needed 226 | if ws_name is not None: 227 | Sheet(ws2reactivate_name, wkb=self.wkb).activate() 228 | 229 | def render(self): 230 | """Main render method.""" 231 | 232 | self.open_template_as_current_wkb() 233 | 234 | save_render_blocks = [block for block in self.render_blocks 235 | if 'save_as' in block.keys()] 236 | 237 | for block in save_render_blocks: 238 | self.apply_render_block(block) 239 | 240 | self.wkb.close() 241 | --------------------------------------------------------------------------------