├── .github └── workflows │ └── tests.yml ├── .gitignore ├── CHANGES.md ├── LICENSE ├── MANIFEST.in ├── README.md ├── mkdocs_diagrams ├── __init__.py └── plugin.py ├── setup.py ├── test_docs └── example.diagrams.py └── test_integration.py /.github/workflows/tests.yml: -------------------------------------------------------------------------------- 1 | name: Run tests 2 | on: [push] 3 | jobs: 4 | integration-tests: 5 | runs-on: ubuntu-latest 6 | steps: 7 | - name: Check out code 8 | uses: actions/checkout@v2 9 | 10 | - name: Set up Python 11 | uses: actions/setup-python@v2 12 | with: 13 | python-version: "3.x" 14 | 15 | - name: Install dependencies 16 | run: pip install . pytest 17 | 18 | - name: Install graphviz 19 | run: sudo apt-get install --no-install-recommends graphviz 20 | 21 | - name: Run tests 22 | run: py.test 23 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | .env 3 | mkdocs_diagrams.egg-info 4 | *.egg-info 5 | build 6 | dist 7 | -------------------------------------------------------------------------------- /CHANGES.md: -------------------------------------------------------------------------------- 1 | # Changelog 2 | 3 | ## v1.0.0 4 | 5 | - Use `sys.executable` when spawning python subprocesses. 6 | - This is primarily a bugfix for Windows users, though other Operating Systems could also benefit. 7 | - Special thanks to [Leonardo Monteiro](https://github.com/decastromonteiro) for helping pinpoint this issue. 8 | - With this plugin now being used successfully in a number of different environments, the version is bumped to 1.0 for a better stability promise. 9 | 10 | ## v0.0.2 11 | 12 | - `diagrams` added as a package dependency. 13 | - Improved error handling. Errors running diagram files are now logged as errors through MkDocs logging handlers. 14 | 15 | ## v0.0.1 16 | 17 | - Initial release. 18 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2020 Nick Groenen 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 4 | 5 | The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. 6 | 7 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 8 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include *.md 2 | include LICENSE 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # mkdocs-diagrams 2 | 3 | A plugin for the [MkDocs] documentation site generator which facilitates easy embedding of system architecture diagrams through the [Diagrams] project ([view examples]). 4 | 5 | ## Installation 6 | 7 | `mkdocs-diagrams` is available on PyPI. 8 | It can be installed through `pip install mkdocs-diagrams` or equivalent command with pipenv or poetry. 9 | 10 | You'll also need to have the [graphviz] `dot` tool installed on your system. 11 | It's available as `graphviz` in most package managers. 12 | 13 | Once installed, configure MkDocs to use this plugin by including `diagrams` in the `plugins` list in your `mkdocs.yml`. 14 | For example: 15 | 16 | ```yaml 17 | plugins: 18 | - diagrams 19 | - search 20 | ``` 21 | 22 | (If you don't have a `plugins` key in your config yet, you'll almost surely want to include `search` as well. 23 | It's a default plugin that will otherwise get deactivated.) 24 | 25 | ## Usage 26 | 27 | > **Warning:** This plugin will execute `.diagram.py` files during build, as that is how [Diagrams] itself operates. 28 | > Be careful using this plugin with untrusted input as this effectively allows arbitrary code execution. 29 | 30 | Once installed, the diagrams plugin can be used by including diagrams files in your docs directory. 31 | 32 | For example, create a file named `example.diagrams.py` with the following contents: 33 | 34 | ```python 35 | from diagrams import Cluster, Diagram 36 | from diagrams.aws.compute import ECS, EKS, Lambda 37 | from diagrams.aws.database import Redshift 38 | from diagrams.aws.integration import SQS 39 | from diagrams.aws.storage import S3 40 | 41 | 42 | with Diagram("Event Processing", show=False): 43 | source = EKS("k8s source") 44 | 45 | with Cluster("Event Flows"): 46 | with Cluster("Event Workers"): 47 | workers = [ECS("worker1"), 48 | ECS("worker2"), 49 | ECS("worker3")] 50 | 51 | queue = SQS("event queue") 52 | 53 | with Cluster("Processing"): 54 | handlers = [Lambda("proc1"), 55 | Lambda("proc2"), 56 | Lambda("proc3")] 57 | 58 | store = S3("events store") 59 | dw = Redshift("analytics") 60 | 61 | source >> workers >> queue >> handlers 62 | handlers >> store 63 | handlers >> dw 64 | ``` 65 | 66 | When MkDocs is run (either with `build` or `serve`), this will result in a file named `event_processing.png` being created. 67 | Include this in your markdown files using regular image syntax: `![Event processing architecture](event_processing.png)` 68 | 69 | ## Configuration 70 | 71 | This plugin supports a few config options, which can be set as follows: 72 | 73 | ```yaml 74 | plugins: 75 | - diagrams: 76 | file_extension: ".diagrams.py" 77 | max_workers: 5 78 | ``` 79 | 80 | ### `file_extension` 81 | 82 | Sets the filename extension for diagram files. 83 | When `mkdocs build` or `mkdocs serve` is run, all files ending in this extension will be executed. 84 | 85 | Default: `.diagrams.py` 86 | 87 | ### `max_workers` 88 | 89 | A pool of workers is used to render diagram files in parallel on multi-core systems. 90 | Setting this allows you to limit the number of workers to this amount. 91 | 92 | Default: Dynamically chosen (`os.cpu_count() + 2`) 93 | 94 | [diagrams]: https://diagrams.mingrammer.com/ 95 | [graphviz]: https://www.graphviz.org/ 96 | [mkdocs]: https://www.mkdocs.org/ 97 | [view examples]: https://diagrams.mingrammer.com/docs/getting-started/examples 98 | -------------------------------------------------------------------------------- /mkdocs_diagrams/__init__.py: -------------------------------------------------------------------------------- 1 | from .plugin import DiagramsPlugin 2 | 3 | __all__ = [DiagramsPlugin] 4 | -------------------------------------------------------------------------------- /mkdocs_diagrams/plugin.py: -------------------------------------------------------------------------------- 1 | import concurrent.futures 2 | import logging 3 | import os 4 | import shutil 5 | import subprocess 6 | import sys 7 | import time 8 | 9 | import mkdocs 10 | import mkdocs.plugins 11 | 12 | from mkdocs.structure.files import get_files 13 | 14 | # This global is a hack to keep track of the last time the plugin rendered diagrams. 15 | # A global is required because plugins are reinitialized each time a change is detected. 16 | last_run_timestamp = 0 17 | 18 | 19 | class DiagramsPlugin(mkdocs.plugins.BasePlugin): 20 | """ 21 | A MkDocs plugin to render Diagrams files. 22 | 23 | See also https://diagrams.mingrammer.com/. 24 | """ 25 | 26 | config_scheme = ( 27 | ( 28 | "file_extension", 29 | mkdocs.config.config_options.Type(str, default=".diagrams.py"), 30 | ), 31 | ("max_workers", mkdocs.config.config_options.Type(int, default=None)), 32 | ) 33 | 34 | def __init__(self): 35 | self.log = logging.getLogger("mkdocs.plugins.diagrams") 36 | self.pool = None 37 | 38 | def _create_threadpool(self): 39 | max_workers = self.config["max_workers"] 40 | if max_workers is None: 41 | max_workers = os.cpu_count() + 2 42 | self.log.debug( 43 | "Using up to %d concurrent workers for diagrams rendering", max_workers 44 | ) 45 | return concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) 46 | 47 | def _render_diagram(self, file): 48 | self.log.debug(f"Rendering {file.name}") 49 | # The two commented lines below would build in the destination 50 | # (site_dir) directory instead of the original source directory. 51 | # Unfortunately this results in incorrect image URLs (they don't get 52 | # rewritten to the proper relative path one directory up). 53 | # 54 | # dest_dir = os.path.dirname(file.abs_dest_path) 55 | # filename = file.abs_dest_path[len(dest_dir)+1:] 56 | dest_dir = os.path.dirname(file.abs_src_path) 57 | filename = file.abs_src_path[len(dest_dir) + 1:] 58 | 59 | # Even when writing in abs_src_path rather than abs_dest_path, this 60 | # seems needed to make livereload accurately pick up changes. 61 | os.makedirs(os.path.dirname(file.abs_dest_path), exist_ok=True) 62 | shutil.copy(file.abs_src_path, file.abs_dest_path) 63 | 64 | # Try to get the full path to the currently used interpreter. This 65 | # helps ensure we use the right one even with virtualenvs, etc., 66 | # especially on Windows. 67 | # (See https://github.com/zoni/mkdocs-diagrams/issues/2) 68 | python_path = sys.executable 69 | 70 | # If Python is unable to retrieve the real path to its 71 | # executable, sys.executable will be an empty string or None. 72 | if python_path is None or python_path == "": 73 | python_path = "python" 74 | 75 | subprocess.run([python_path, filename], check=True, cwd=dest_dir) 76 | 77 | def _walk_files_and_render(self, config): 78 | pool = self._create_threadpool() 79 | files = get_files(config) 80 | jobs = [] 81 | for file in files: 82 | if file.src_path.endswith(self.config["file_extension"]): 83 | jobs.append(pool.submit(self._render_diagram, file)) 84 | 85 | for job in concurrent.futures.as_completed(jobs): 86 | try: 87 | job.result() 88 | except Exception: 89 | self.log.exception("Worker raised an exception while rendering a diagram") 90 | 91 | def on_pre_build(self, config): 92 | global last_run_timestamp 93 | if int(time.time()) - last_run_timestamp < 10: 94 | self.log.info( 95 | "Watcher started looping, skipping diagrams rendering on this run" 96 | ) 97 | return 98 | self._walk_files_and_render(config) 99 | last_run_timestamp = int(time.time()) 100 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import os.path 2 | import setuptools 3 | 4 | 5 | def read(name): 6 | mydir = os.path.abspath(os.path.dirname(__file__)) 7 | return open(os.path.join(mydir, name)).read() 8 | 9 | 10 | setuptools.setup( 11 | name="mkdocs-diagrams", 12 | version="1.0.0", 13 | packages=["mkdocs_diagrams"], 14 | url="https://github.com/zoni/mkdocs-diagrams", 15 | license="MIT", 16 | author="Nick Groenen", 17 | author_email="nick@groenen.me", 18 | description="MkDocs plugin to render Diagrams files", 19 | long_description=read("README.md"), 20 | long_description_content_type="text/markdown", 21 | install_requires=["mkdocs", "diagrams"], 22 | entry_points={"mkdocs.plugins": ["diagrams = mkdocs_diagrams:DiagramsPlugin",]}, 23 | classifiers=[ 24 | "Development Status :: 5 - Production/Stable", 25 | "License :: OSI Approved :: MIT License", 26 | "Operating System :: OS Independent", 27 | "Programming Language :: Python :: 3", 28 | ], 29 | ) 30 | -------------------------------------------------------------------------------- /test_docs/example.diagrams.py: -------------------------------------------------------------------------------- 1 | from diagrams import Cluster, Diagram 2 | from diagrams.aws.compute import ECS, EKS, Lambda 3 | from diagrams.aws.database import Redshift 4 | from diagrams.aws.integration import SQS 5 | from diagrams.aws.storage import S3 6 | 7 | 8 | with Diagram("Event Processing", show=False): 9 | source = EKS("k8s source") 10 | 11 | with Cluster("Event Flows"): 12 | with Cluster("Event Workers"): 13 | workers = [ECS("worker1"), 14 | ECS("worker2"), 15 | ECS("worker3")] 16 | 17 | queue = SQS("event queue") 18 | 19 | with Cluster("Processing"): 20 | handlers = [Lambda("proc1"), 21 | Lambda("proc2"), 22 | Lambda("proc3")] 23 | 24 | store = S3("events store") 25 | dw = Redshift("analytics") 26 | 27 | source >> workers >> queue >> handlers 28 | handlers >> store 29 | handlers >> dw 30 | -------------------------------------------------------------------------------- /test_integration.py: -------------------------------------------------------------------------------- 1 | import shutil 2 | import subprocess 3 | import os 4 | 5 | MKDOCS_CONFIG = """ 6 | site_name: mkdocs-diagrams 7 | docs_dir: docs 8 | 9 | plugins: 10 | - diagrams 11 | """ 12 | 13 | 14 | def file_contents(path): 15 | with open(path, 'r') as f: 16 | return f.read() 17 | 18 | 19 | def test_build(tmp_path): 20 | with open(tmp_path / "mkdocs.yml", "w") as f: 21 | f.write(MKDOCS_CONFIG) 22 | shutil.copytree("test_docs", tmp_path / "docs") 23 | shutil.copy("README.md", tmp_path / "docs" / "README.md") 24 | 25 | subprocess.run( 26 | ["mkdocs", "build", "--verbose", "--strict", "--site-dir", tmp_path / "site"], 27 | check=True, 28 | cwd=tmp_path 29 | ) 30 | 31 | index_html = file_contents(tmp_path / "site" / "index.html") 32 | assert "mkdocs-diagrams" in index_html 33 | assert os.path.exists(tmp_path / "site" / "event_processing.png") 34 | --------------------------------------------------------------------------------