├── LICENSE ├── Makefile ├── README.md ├── expected-auto.native ├── expected.native ├── file-a.md ├── file-b.md ├── file-c.md ├── file-d.org ├── file-f.md ├── include-files.lua ├── sample.md └── subdir └── file-g.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Pandoc Extensions 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | DIFF ?= diff --strip-trailing-cr -u 2 | PANDOC ?= pandoc 3 | 4 | test: sample.md file-a.md file-b.md file-c.md include-files.lua 5 | @$(PANDOC) --lua-filter=include-files.lua --to=native $< \ 6 | | $(DIFF) expected.native - 7 | @$(PANDOC) --lua-filter=include-files.lua -M include-auto --to=native $< \ 8 | | $(DIFF) expected-auto.native - 9 | 10 | expected.native: sample.md file-a.md file-b.md file-c.md include-files.lua 11 | $(PANDOC) --lua-filter=include-files.lua --output $@ $< 12 | 13 | expected-auto.native: sample.md file-a.md file-b.md file-c.md include-files.lua 14 | $(PANDOC) --lua-filter=include-files.lua -M include-auto --output $@ $< 15 | 16 | .PHONY: test 17 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # include-files 2 | 3 | Filter to include other files in the document. 4 | 5 | ## Note 6 | 7 | Do not run other filters before executing `include-files`, unless you are 8 | certain the other filter does not require `include-files` to function properly. 9 | Executing most filters without first using `include-files` will result in the 10 | other filter seeing an empty code block. 11 | 12 | ## Usage 13 | 14 | Use a special code block with class `include` to include files of the same 15 | format as the input. Each code line is treated as the filename of a file, 16 | parsed, and the result is added to the document. 17 | 18 | Metadata from included files is discarded. 19 | 20 | ### Shifting Headings 21 | 22 | The default is to include the subdocuments unchanged, but it can be convenient 23 | to modify the level of headers; a top-level header in an included file should be 24 | a second or third-level header in the final document. 25 | 26 | #### Manual shifting 27 | 28 | Use the `shift-heading-level-by` attribute to control header shifting. 29 | 30 | #### Automatic shifting 31 | 32 | 1. Add metadata `-M include-auto` to enable automatic shifting. 33 | 2. Do not specify `shift-heading-level-by` 34 | 3. It will be inferred to the last heading level encountered 35 | 36 | _Example_ : 37 | 38 | ````md 39 | # Title f 40 | 41 | This is `file-f.md`. 42 | 43 | ## Subtitle f 44 | 45 | ```{.include} >> equivalent to {.include shift-heading-level-by=2} 46 | file-a.md 47 | ``` 48 | 49 | ```{.include shift-heading-level-by=1} >> force shift to be 1 50 | file-a.md 51 | ``` 52 | ```` 53 | 54 | ### Comments 55 | 56 | Comment lines can be added in the include block by beginning a line with two 57 | `//` characters. 58 | 59 | ### Different formats 60 | 61 | Files are assumed to be written in Markdown, but sometimes one will want to 62 | include files written in a different format. An alternative format can be 63 | specified via the `format` attribute. Only plain-text formats are accepted. 64 | 65 | ### Recursive transclusion 66 | 67 | Included files can in turn include other files. Note that all filenames must be 68 | relative to the directory from which they are included. I.e., if a file `a/b.md` 69 | is included in the main document, and another file `a/b/c.md` should be included 70 | from `a/b.md`, then the relative path from `a/b.md` must be used, in this case 71 | `b/c.md`. The full relative path will be automatically generated in the final 72 | document. The same goes for image paths, link paths and codeblock file paths 73 | using the `include-code-files` filter. 74 | 75 | ## Example 76 | 77 | Let's assume we are writing a longer document, like a thesis. Each chapter and 78 | appendix section resides in its own file, with some additional information in 79 | the main file `main.md`: 80 | 81 | --- 82 | author: me 83 | title: Thesis 84 | --- 85 | 86 | # Frontmatter 87 | 88 | Thanks everyone! 89 | 90 | 91 | 92 | ``` {.include} 93 | chapters/introduction.md 94 | chapters/methods.md 95 | chapters/results.md 96 | chapters/discussion.md 97 | ``` 98 | 99 | # Appendix 100 | 101 | More info goes here. 102 | 103 | ``` {.include shift-heading-level-by=1} 104 | // headings in included documents are shifted down a level, 105 | // a level 1 heading becomes level 2. 106 | appendix/questionaire.md 107 | ``` 108 | 109 | An HTML can be produced with this command: 110 | 111 | pandoc --lua-filter=include-files.lua main.md --output result.html 112 | -------------------------------------------------------------------------------- /expected-auto.native: -------------------------------------------------------------------------------- 1 | [ Header 2 | 1 3 | ( "section-1" , [] , [] ) 4 | [ Str "Section" , Space , Str "1" ] 5 | , Para [ Str "Thanks" , Space , Str "everyone!" ] 6 | , Header 7 | 2 8 | ( "title-of-file-a" , [] , [] ) 9 | [ Str "Title" , Space , Str "of" , Space , Str "file-a" ] 10 | , Para 11 | [ Str "This" 12 | , Space 13 | , Str "is" 14 | , Space 15 | , Code ( "" , [] , [] ) "file-a.md" 16 | , Str "." 17 | ] 18 | , Header 19 | 2 ( "file-b" , [] , [] ) [ Str "File" , Space , Str "b" ] 20 | , Para 21 | [ Str "This" 22 | , Space 23 | , Str "is" 24 | , Space 25 | , Code ( "" , [] , [] ) "file-b.md" 26 | , Str "." 27 | ] 28 | , Para 29 | [ Str "It" 30 | , Space 31 | , Str "has" 32 | , Space 33 | , Str "two" 34 | , Space 35 | , Str "paragraphs" 36 | , Space 37 | , Str "and" 38 | , Space 39 | , Str "a" 40 | , Space 41 | , Str "header." 42 | ] 43 | , Header 44 | 1 45 | ( "different-format" , [] , [] ) 46 | [ Str "Different" , Space , Str "format" ] 47 | , Header 48 | 2 49 | ( "org-header" , [] , [] ) 50 | [ Str "Org" , Space , Str "header" ] 51 | , Para 52 | [ Str "This" 53 | , Space 54 | , Str "is" 55 | , Space 56 | , Emph [ Str "emphasized" ] 57 | , Str "." 58 | ] 59 | , Header 60 | 1 61 | ( "recursive-transclusion" , [] , [] ) 62 | [ Str "Recursive" , Space , Str "transclusion" ] 63 | , Header 64 | 2 ( "title-f" , [] , [] ) [ Str "Title" , Space , Str "f" ] 65 | , Para 66 | [ Str "This" 67 | , Space 68 | , Str "is" 69 | , Space 70 | , Code ( "" , [] , [] ) "file-f.md" 71 | , Str "." 72 | ] 73 | , Header 74 | 3 75 | ( "subtitle-f" , [] , [] ) 76 | [ Str "Subtitle" , Space , Str "f" ] 77 | , Header 78 | 4 79 | ( "title-of-file-a" , [] , [] ) 80 | [ Str "Title" , Space , Str "of" , Space , Str "file-a" ] 81 | , Para 82 | [ Str "This" 83 | , Space 84 | , Str "is" 85 | , Space 86 | , Code ( "" , [] , [] ) "file-a.md" 87 | , Str "." 88 | ] 89 | , Header 90 | 3 91 | ( "title-of-file-a" , [] , [] ) 92 | [ Str "Title" , Space , Str "of" , Space , Str "file-a" ] 93 | , Para 94 | [ Str "This" 95 | , Space 96 | , Str "is" 97 | , Space 98 | , Code ( "" , [] , [] ) "file-a.md" 99 | , Str "." 100 | ] 101 | , Header 102 | 1 ( "subdirectories" , [] , [] ) [ Str "Subdirectories" ] 103 | , Header 104 | 2 105 | ( "image-include" , [] , [] ) 106 | [ Str "Image" , Space , Str "include" ] 107 | , Para 108 | [ Str "Image" 109 | , Space 110 | , Str "relative" 111 | , Space 112 | , Str "path" 113 | , Space 114 | , Str "will" 115 | , Space 116 | , Str "be" 117 | , Space 118 | , Str "updated." 119 | ] 120 | , Figure 121 | ( "" , [] , [] ) 122 | (Caption 123 | Nothing [ Plain [ Str "Image" , Space , Str "title" ] ]) 124 | [ Plain 125 | [ Image 126 | ( "" , [] , [] ) 127 | [ Str "Image" , Space , Str "title" ] 128 | ( "subdir/someimage.png" , "" ) 129 | ] 130 | ] 131 | , Para 132 | [ Link 133 | ( "" , [] , [] ) 134 | [ Str "Some" , Space , Str "link" ] 135 | ( "subdir/someimage.png" , "" ) 136 | ] 137 | , Header 138 | 2 139 | ( "source-include" , [] , [] ) 140 | [ Str "Source" , Space , Str "include" ] 141 | , Para 142 | [ Str "File" 143 | , Space 144 | , Str "inclusion" 145 | , Space 146 | , Str "codeblocks" 147 | , Space 148 | , Str "for" 149 | , Space 150 | , Str "use" 151 | , Space 152 | , Str "with" 153 | , Space 154 | , Str "include-code-files" 155 | , Space 156 | , Str "will" 157 | , Space 158 | , Str "be" 159 | , SoftBreak 160 | , Str "updated" 161 | , Space 162 | , Str "too." 163 | ] 164 | , CodeBlock 165 | ( "" , [ "c" ] , [ ( "include" , "subdir/somecode.c" ) ] ) 166 | "" 167 | , Header 1 ( "appendix" , [] , [] ) [ Str "Appendix" ] 168 | , Para 169 | [ Str "More" 170 | , Space 171 | , Str "info" 172 | , Space 173 | , Str "goes" 174 | , Space 175 | , Str "here." 176 | ] 177 | , Header 178 | 2 ( "questionaire" , [] , [] ) [ Str "Questionaire" ] 179 | , BulletList 180 | [ [ Plain 181 | [ Str "Is" , Space , Str "this" , Space , Str "good?" ] 182 | ] 183 | ] 184 | ] 185 | -------------------------------------------------------------------------------- /expected.native: -------------------------------------------------------------------------------- 1 | [ Header 2 | 1 3 | ( "section-1" , [] , [] ) 4 | [ Str "Section" , Space , Str "1" ] 5 | , Para [ Str "Thanks" , Space , Str "everyone!" ] 6 | , Header 7 | 1 8 | ( "title-of-file-a" , [] , [] ) 9 | [ Str "Title" , Space , Str "of" , Space , Str "file-a" ] 10 | , Para 11 | [ Str "This" 12 | , Space 13 | , Str "is" 14 | , Space 15 | , Code ( "" , [] , [] ) "file-a.md" 16 | , Str "." 17 | ] 18 | , Header 19 | 1 ( "file-b" , [] , [] ) [ Str "File" , Space , Str "b" ] 20 | , Para 21 | [ Str "This" 22 | , Space 23 | , Str "is" 24 | , Space 25 | , Code ( "" , [] , [] ) "file-b.md" 26 | , Str "." 27 | ] 28 | , Para 29 | [ Str "It" 30 | , Space 31 | , Str "has" 32 | , Space 33 | , Str "two" 34 | , Space 35 | , Str "paragraphs" 36 | , Space 37 | , Str "and" 38 | , Space 39 | , Str "a" 40 | , Space 41 | , Str "header." 42 | ] 43 | , Header 44 | 1 45 | ( "different-format" , [] , [] ) 46 | [ Str "Different" , Space , Str "format" ] 47 | , Header 48 | 2 49 | ( "org-header" , [] , [] ) 50 | [ Str "Org" , Space , Str "header" ] 51 | , Para 52 | [ Str "This" 53 | , Space 54 | , Str "is" 55 | , Space 56 | , Emph [ Str "emphasized" ] 57 | , Str "." 58 | ] 59 | , Header 60 | 1 61 | ( "recursive-transclusion" , [] , [] ) 62 | [ Str "Recursive" , Space , Str "transclusion" ] 63 | , Header 64 | 1 ( "title-f" , [] , [] ) [ Str "Title" , Space , Str "f" ] 65 | , Para 66 | [ Str "This" 67 | , Space 68 | , Str "is" 69 | , Space 70 | , Code ( "" , [] , [] ) "file-f.md" 71 | , Str "." 72 | ] 73 | , Header 74 | 2 75 | ( "subtitle-f" , [] , [] ) 76 | [ Str "Subtitle" , Space , Str "f" ] 77 | , Header 78 | 1 79 | ( "title-of-file-a" , [] , [] ) 80 | [ Str "Title" , Space , Str "of" , Space , Str "file-a" ] 81 | , Para 82 | [ Str "This" 83 | , Space 84 | , Str "is" 85 | , Space 86 | , Code ( "" , [] , [] ) "file-a.md" 87 | , Str "." 88 | ] 89 | , Header 90 | 2 91 | ( "title-of-file-a" , [] , [] ) 92 | [ Str "Title" , Space , Str "of" , Space , Str "file-a" ] 93 | , Para 94 | [ Str "This" 95 | , Space 96 | , Str "is" 97 | , Space 98 | , Code ( "" , [] , [] ) "file-a.md" 99 | , Str "." 100 | ] 101 | , Header 102 | 1 ( "subdirectories" , [] , [] ) [ Str "Subdirectories" ] 103 | , Header 104 | 1 105 | ( "image-include" , [] , [] ) 106 | [ Str "Image" , Space , Str "include" ] 107 | , Para 108 | [ Str "Image" 109 | , Space 110 | , Str "relative" 111 | , Space 112 | , Str "path" 113 | , Space 114 | , Str "will" 115 | , Space 116 | , Str "be" 117 | , Space 118 | , Str "updated." 119 | ] 120 | , Figure 121 | ( "" , [] , [] ) 122 | (Caption 123 | Nothing [ Plain [ Str "Image" , Space , Str "title" ] ]) 124 | [ Plain 125 | [ Image 126 | ( "" , [] , [] ) 127 | [ Str "Image" , Space , Str "title" ] 128 | ( "subdir/someimage.png" , "" ) 129 | ] 130 | ] 131 | , Para 132 | [ Link 133 | ( "" , [] , [] ) 134 | [ Str "Some" , Space , Str "link" ] 135 | ( "subdir/someimage.png" , "" ) 136 | ] 137 | , Header 138 | 1 139 | ( "source-include" , [] , [] ) 140 | [ Str "Source" , Space , Str "include" ] 141 | , Para 142 | [ Str "File" 143 | , Space 144 | , Str "inclusion" 145 | , Space 146 | , Str "codeblocks" 147 | , Space 148 | , Str "for" 149 | , Space 150 | , Str "use" 151 | , Space 152 | , Str "with" 153 | , Space 154 | , Str "include-code-files" 155 | , Space 156 | , Str "will" 157 | , Space 158 | , Str "be" 159 | , SoftBreak 160 | , Str "updated" 161 | , Space 162 | , Str "too." 163 | ] 164 | , CodeBlock 165 | ( "" , [ "c" ] , [ ( "include" , "subdir/somecode.c" ) ] ) 166 | "" 167 | , Header 1 ( "appendix" , [] , [] ) [ Str "Appendix" ] 168 | , Para 169 | [ Str "More" 170 | , Space 171 | , Str "info" 172 | , Space 173 | , Str "goes" 174 | , Space 175 | , Str "here." 176 | ] 177 | , Header 178 | 2 ( "questionaire" , [] , [] ) [ Str "Questionaire" ] 179 | , BulletList 180 | [ [ Plain 181 | [ Str "Is" , Space , Str "this" , Space , Str "good?" ] 182 | ] 183 | ] 184 | ] 185 | -------------------------------------------------------------------------------- /file-a.md: -------------------------------------------------------------------------------- 1 | # Title of file-a 2 | 3 | This is `file-a.md`. 4 | -------------------------------------------------------------------------------- /file-b.md: -------------------------------------------------------------------------------- 1 | # File b 2 | 3 | This is `file-b.md`. 4 | 5 | It has two paragraphs and a header. 6 | -------------------------------------------------------------------------------- /file-c.md: -------------------------------------------------------------------------------- 1 | # Questionaire 2 | 3 | - Is this good? 4 | -------------------------------------------------------------------------------- /file-d.org: -------------------------------------------------------------------------------- 1 | * Org header 2 | 3 | This is /emphasized/. 4 | -------------------------------------------------------------------------------- /file-f.md: -------------------------------------------------------------------------------- 1 | # Title f 2 | 3 | This is `file-f.md`. 4 | 5 | ## Subtitle f 6 | 7 | ```{.include} 8 | file-a.md 9 | ``` 10 | 11 | ```{.include shift-heading-level-by=1} 12 | file-a.md 13 | ``` 14 | -------------------------------------------------------------------------------- /include-files.lua: -------------------------------------------------------------------------------- 1 | --- include-files.lua – filter to include Markdown files 2 | --- 3 | --- Copyright: © 2019–2021 Albert Krewinkel 4 | --- License: MIT – see LICENSE file for details 5 | 6 | -- Module pandoc.path is required and was added in version 2.12 7 | PANDOC_VERSION:must_be_at_least '2.12' 8 | 9 | local List = require 'pandoc.List' 10 | local path = require 'pandoc.path' 11 | local system = require 'pandoc.system' 12 | 13 | --- Get include auto mode 14 | local include_auto = false 15 | function get_vars (meta) 16 | if meta['include-auto'] then 17 | include_auto = true 18 | end 19 | end 20 | 21 | --- Keep last heading level found 22 | local last_heading_level = 0 23 | function update_last_level(header) 24 | last_heading_level = header.level 25 | end 26 | 27 | --- Update contents of included file 28 | local function update_contents(blocks, shift_by, include_path) 29 | local update_contents_filter = { 30 | -- Shift headings in block list by given number 31 | Header = function (header) 32 | if shift_by then 33 | header.level = header.level + shift_by 34 | end 35 | return header 36 | end, 37 | -- If link paths are relative then prepend include file path 38 | Link = function (link) 39 | if path.is_relative(link.target) and string.sub(path.filename(link.target), 1, 1) ~= '#' then 40 | link.target = path.normalize(path.join({include_path, link.target})) 41 | end 42 | return link 43 | end, 44 | -- If image paths are relative then prepend include file path 45 | Image = function (image) 46 | if path.is_relative(image.src) then 47 | image.src = path.normalize(path.join({include_path, image.src})) 48 | end 49 | return image 50 | end, 51 | -- Update path for include-code-files.lua filter style CodeBlocks 52 | CodeBlock = function (cb) 53 | if cb.attributes.include and path.is_relative(cb.attributes.include) then 54 | cb.attributes.include = 55 | path.normalize(path.join({include_path, cb.attributes.include})) 56 | end 57 | return cb 58 | end 59 | } 60 | 61 | return pandoc.walk_block(pandoc.Div(blocks), update_contents_filter).content 62 | end 63 | 64 | --- Filter function for code blocks 65 | local transclude 66 | function transclude (cb) 67 | -- ignore code blocks which are not of class "include". 68 | if not cb.classes:includes 'include' then 69 | return 70 | end 71 | 72 | -- Markdown is used if this is nil. 73 | local format = cb.attributes['format'] 74 | 75 | -- Attributes shift headings 76 | local shift_heading_level_by = 0 77 | local shift_input = cb.attributes['shift-heading-level-by'] 78 | if shift_input then 79 | shift_heading_level_by = tonumber(shift_input) 80 | else 81 | if include_auto then 82 | -- Auto shift headings 83 | shift_heading_level_by = last_heading_level 84 | end 85 | end 86 | 87 | --- keep track of level before recusion 88 | local buffer_last_heading_level = last_heading_level 89 | 90 | local blocks = List:new() 91 | for line in cb.text:gmatch('[^\n]+') do 92 | if line:sub(1,2) ~= '//' then 93 | local fh = io.open(line) 94 | if not fh then 95 | io.stderr:write("Cannot open file " .. line .. " | Skipping includes\n") 96 | else 97 | -- read file as the given format with global reader options 98 | local contents = pandoc.read( 99 | fh:read '*a', 100 | format, 101 | PANDOC_READER_OPTIONS 102 | ).blocks 103 | last_heading_level = 0 104 | -- recursive transclusion 105 | contents = system.with_working_directory( 106 | path.directory(line), 107 | function () 108 | return pandoc.walk_block( 109 | pandoc.Div(contents), 110 | { Header = update_last_level, CodeBlock = transclude } 111 | ) 112 | end).content 113 | --- reset to level before recursion 114 | last_heading_level = buffer_last_heading_level 115 | blocks:extend(update_contents(contents, shift_heading_level_by, 116 | path.directory(line))) 117 | fh:close() 118 | end 119 | end 120 | end 121 | return blocks 122 | end 123 | 124 | return { 125 | { Meta = get_vars }, 126 | { Header = update_last_level, CodeBlock = transclude } 127 | } 128 | -------------------------------------------------------------------------------- /sample.md: -------------------------------------------------------------------------------- 1 | --- 2 | author: me 3 | title: Thesis 4 | --- 5 | 6 | # Section 1 7 | 8 | Thanks everyone! 9 | 10 | ``` {.include} 11 | // file-a has just a paragraph 12 | file-a.md 13 | // file-b contains a header 14 | file-b.md 15 | ``` 16 | 17 | # Different format 18 | 19 | ``` {.include format=org shift-heading-level-by=1} 20 | // org-mode file 21 | file-d.org 22 | ``` 23 | 24 | # Recursive transclusion 25 | 26 | ``` {.include} 27 | // this will also include file-a.md 28 | file-f.md 29 | ``` 30 | 31 | # Subdirectories 32 | 33 | ``` {.include} 34 | // file-g.md includes an image and source code. The relative 35 | // path used in file-g.md will be prefixed with subdir in the 36 | // final document. 37 | subdir/file-g.md 38 | ``` 39 | 40 | # Appendix 41 | 42 | More info goes here. 43 | 44 | ``` {.include shift-heading-level-by=1} 45 | // headings in included documents are shifted down a level, 46 | // a level 1 heading becomes level 2. 47 | file-c.md 48 | ``` 49 | -------------------------------------------------------------------------------- /subdir/file-g.md: -------------------------------------------------------------------------------- 1 | # Image include 2 | 3 | Image relative path will be updated. 4 | 5 | ![Image title](someimage.png) 6 | 7 | [Some link](someimage.png) 8 | 9 | # Source include 10 | 11 | File inclusion codeblocks for use with include-code-files will be 12 | updated too. 13 | 14 | ```{.c include=somecode.c} 15 | ``` 16 | 17 | --------------------------------------------------------------------------------