├── .github └── workflows │ └── build.yml ├── Dockerfile ├── LICENSE ├── README.md ├── codedown.gif ├── codedown.js ├── lib └── codedown.js ├── package.json └── test ├── codedown.js └── task-list.js /.github/workflows/build.yml: -------------------------------------------------------------------------------- 1 | name: build 2 | 3 | on: [push, pull_request_target] 4 | 5 | jobs: 6 | build: 7 | 8 | runs-on: ubuntu-latest 9 | 10 | strategy: 11 | matrix: 12 | node-version: [12.x, 14.x, 16.x, 17.x] # https://nodejs.org/en/about/releases/#releases 13 | 14 | steps: 15 | - uses: actions/checkout@v1 16 | - name: Use Node.js ${{ matrix.node-version }} 17 | uses: actions/setup-node@v1 18 | with: 19 | node-version: ${{ matrix.node-version }} 20 | - name: npm install, build, and test 21 | run: | 22 | npm install 23 | npm ci 24 | npm run build --if-present 25 | echo '{ "laxcomma": true }' > .jshintrc 26 | npm test 27 | npm run coverage 28 | npm run coveralls 29 | env: 30 | CI: true 31 | COVERALLS_REPO_TOKEN: ${{ secrets.COVERALLS_REPO_TOKEN }} 32 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM node:lts-slim 2 | 3 | COPY . /app 4 | RUN npm install -g codedown 5 | 6 | ENTRYPOINT [ "codedown" ] -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2016 James Earl Douglas 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a 4 | copy of this software and associated documentation files (the 5 | "Software"), to deal in the Software without restriction, including 6 | without limitation the rights to use, copy, modify, merge, publish, 7 | distribute, sublicense, and/or sell copies of the Software, and to 8 | permit persons to whom the Software is furnished to do so, subject to 9 | the following conditions: 10 | 11 | The above copyright notice and this permission notice shall be included 12 | in all copies or substantial portions of the Software. 13 | 14 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS 15 | OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 16 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. 17 | IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY 18 | CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, 19 | TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 20 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 21 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Build Status][build-badge]][build-link] 2 | [![Coverage Status][coverage-badge]][coverage-link] 3 | [![npm version][release-badge]][release-link] 4 | 5 | [build-badge]: https://github.com/earldouglas/codedown/workflows/build/badge.svg 6 | [build-link]: https://github.com/earldouglas/codedown/actions 7 | [coverage-badge]: https://coveralls.io/repos/github/earldouglas/codedown/badge.svg 8 | [coverage-link]: https://coveralls.io/github/earldouglas/codedown 9 | [release-badge]: https://badge.fury.io/js/codedown.svg 10 | [release-link]: https://www.npmjs.com/package/codedown 11 | 12 | # Codedown 13 | 14 | Codedown is a little utility to extract code blocks from Markdown files. 15 | Inspired by [literate 16 | Haskell](https://wiki.haskell.org/Literate_programming), Codedown can be 17 | used to: 18 | 19 | * Validate the correctness of code embedded in Markdown 20 | * Run code embedded in Markdown 21 | * Ship code and Markdown together in harmony 22 | 23 | ![](codedown.gif) 24 | 25 | ## Quicker start 26 | 27 | To skip installing Codedown locally, [try it 28 | online](https://earldouglas.github.io/codedown/). 29 | 30 | ## Quick start 31 | 32 | Install Codedown: 33 | 34 | ``` 35 | $ npm install -g codedown 36 | ``` 37 | 38 | Run Codedown: 39 | 40 | ``` 41 | Usage: codedown [...] 42 | 43 | Options: 44 | --separator 45 | --section
46 | 47 | Example: 48 | cat README.md | codedown haskell --separator=----- --section 1.3 49 | ``` 50 | 51 | Codedown reads Markdown from stin, extracts the code blocks designated 52 | as language ``, and outputs them to stdout. The example above 53 | extracts the Haskell code from section 1.3 of this file, and outputs it 54 | with five dashes in between each block: 55 | 56 | ``` 57 | x :: Int 58 | x = 42 59 | ----- 60 | main :: IO () 61 | main = putStrLn $ show x 62 | ``` 63 | 64 | We can pipe the output of Codedown to a language interpreter: 65 | 66 | ``` 67 | $ cat README.md | codedown haskell | runhaskell 68 | 42 69 | ``` 70 | 71 | ``` 72 | $ cat README.md | codedown javascript | node 73 | 42 74 | ``` 75 | 76 | ``` 77 | $ cat README.md | codedown scala | xargs -0 scala -e 78 | 42 79 | ``` 80 | 81 | ## Examples 82 | 83 | This readme is a Markdown file, so we can use Codedown to extract code 84 | from it. 85 | 86 | ### Variables in different languages 87 | 88 | In the following code blocks, let's set `x` to 42 in different 89 | languages: 90 | 91 | *Haskell:* 92 | 93 | ```haskell 94 | x :: Int 95 | x = 42 96 | ``` 97 | 98 | *JavaScript:* 99 | 100 | ```javascript 101 | var x = 42; 102 | ``` 103 | 104 | *Scala:* 105 | 106 | ```scala 107 | val x = 42 108 | ``` 109 | 110 | ### Console output in different languages 111 | 112 | Now let's print `x` it to stdout in different languages. This time, the 113 | code blocks are nested within an unordered list: 114 | 115 | * *Haskell:* 116 | 117 | ```haskell 118 | main :: IO () 119 | main = putStrLn $ show x 120 | ``` 121 | 122 | * *JavaScript:* 123 | 124 | ```javascript 125 | console.log(x); 126 | ``` 127 | 128 | * *Scala:* 129 | 130 | ```scala 131 | println(x) 132 | ``` 133 | 134 | ### Docker 135 | 136 | Build and run a Docker image: 137 | 138 | ``` 139 | docker build -t codedown:dev . 140 | ``` 141 | 142 | Use it to extract `haskell` code blocks and save to `output.hs`: 143 | 144 | ``` 145 | cat README.md | docker run -i codedown:dev haskell > output.hs 146 | ``` 147 | 148 | ## Sections and subsections 149 | 150 | The section above is 1.3, counting by headings. It has two subsections 151 | (1.3.1 and 1.3.2). We can specify a section number to extract the 152 | content from just that section: 153 | 154 | ``` 155 | $ cat README.md | codedown haskell --section 1.3 156 | x :: Int 157 | x = 42 158 | 159 | main :: IO () 160 | main = putStrLn $ show x 161 | ``` 162 | 163 | ``` 164 | $ cat README.md | codedown haskell --section 1.3.1 165 | x :: Int 166 | x = 42 167 | ``` 168 | 169 | ``` 170 | $ cat README.md | codedown haskell --section 1.3.2 171 | main :: IO () 172 | main = putStrLn $ show x 173 | ``` 174 | 175 | We can also specify a section by heading: 176 | 177 | ``` 178 | cat README.md | ./codedown.js haskell --section '### Variables in different languages' 179 | x :: Int 180 | x = 42 181 | ``` 182 | 183 | ## Wildcard matching 184 | 185 | Codedown can use wildcards to match file paths, which are used by some 186 | markdown implementations: 187 | 188 | *lib/codedown.js:* 189 | 190 | ```lib/codedown.js 191 | var x = 42; 192 | ``` 193 | 194 | ``` 195 | $ cat README.md | codedown '**/*.js' 196 | var x = 42 197 | ``` 198 | 199 | Additionally, you can use a special `*` character in place of the language 200 | option to extract any/all code blocks agnostic of language: 201 | 202 | ``` 203 | $ cat README.md | codedown '*' 204 | ``` 205 | 206 | ## Separator 207 | 208 | If there are multiple code blocks in the same file, we can specify a 209 | separator to insert in between them: 210 | 211 | ``` 212 | $ cat README.md | codedown haskell --separator=----- 213 | x :: Int 214 | x = 42 215 | ----- 216 | main :: IO () 217 | main = putStrLn $ show x 218 | ``` 219 | -------------------------------------------------------------------------------- /codedown.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/earldouglas/codedown/d86314ddfeeca94b0380002da465f458b913ccbb/codedown.gif -------------------------------------------------------------------------------- /codedown.js: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env node 2 | 3 | var readline = require('readline'); 4 | var codedown = require('./lib/codedown.js'); 5 | var arg = require('arg'); 6 | 7 | var args = 8 | arg({ 9 | '--separator': String, 10 | '--section': String, 11 | }); 12 | 13 | if (process.argv.length >= 3) { 14 | 15 | var source = []; 16 | 17 | readline.createInterface({ 18 | terminal: false, 19 | input: process.stdin, 20 | }).on('line', function (line) { 21 | source.push(line); 22 | }).on('close', function () { 23 | var lang = process.argv[2]; 24 | var separator = args['--separator']; 25 | var section = args['--section']; 26 | output = codedown(source.join('\n'), lang, separator, section); 27 | console.log(output); 28 | }); 29 | 30 | } else { 31 | console.log('Usage: codedown [...]'); 32 | console.log(''); 33 | console.log('Options:'); 34 | console.log('--separator '); 35 | console.log('--section
'); 36 | console.log(''); 37 | console.log('Example:'); 38 | console.log('cat README.md | codedown haskell --separator=----- --section 1.3'); 39 | } 40 | -------------------------------------------------------------------------------- /lib/codedown.js: -------------------------------------------------------------------------------- 1 | (function(root) { 2 | 3 | 'use strict'; 4 | 5 | var marked = root.marked || require('marked'); 6 | var minimatch = root.minimatch || require("minimatch"); 7 | 8 | var codedown = function(src, lang, separator, targetSection) { 9 | 10 | if (separator === undefined) { 11 | separator = ''; 12 | } 13 | 14 | separator = separator + '\n'; 15 | 16 | var renderer = new marked.Renderer(); 17 | 18 | var renderers = 19 | Object.getOwnPropertyNames(marked.Renderer.prototype); 20 | 21 | for (var i = 0; i < renderers.length; i++) { 22 | var f = renderers[i]; 23 | if (f !== 'constructor') { 24 | renderer[renderers[i]] = function () { return ''; }; 25 | } 26 | } 27 | 28 | var currentSectionNumber = []; 29 | var currentSectionName = null; 30 | var sectionNumbers = {}; 31 | 32 | var sectionNumberMatches = 33 | function () { 34 | return currentSectionNumber.join('.').startsWith(targetSection); 35 | }; 36 | 37 | var sectionNameMatches = 38 | function () { 39 | return sectionNumbers[targetSection] && 40 | sectionNumbers[currentSectionName].startsWith(sectionNumbers[targetSection]); 41 | }; 42 | 43 | var sectionMatches = 44 | function () { 45 | return !targetSection || sectionNumberMatches() || sectionNameMatches(); 46 | }; 47 | 48 | var languageMatches = 49 | function (language) { 50 | return language && (language === '*' || minimatch(language, lang)); 51 | }; 52 | 53 | renderer.heading = 54 | function(text, level, raw) { 55 | var index = level - 1; // 0-based indexing 56 | for (var i = 0; i <= index; i++) { 57 | currentSectionNumber[i] = currentSectionNumber[i] || 0; 58 | } 59 | currentSectionNumber[index] = currentSectionNumber[index] + 1; 60 | currentSectionNumber.splice(level); 61 | 62 | currentSectionName = ('#'.repeat(level) + ' ' + raw).trim(); 63 | sectionNumbers[currentSectionName] = currentSectionNumber.join('.'); 64 | 65 | return ''; 66 | }; 67 | 68 | renderer.code = 69 | function (src, language, escaped) { 70 | 71 | var result = ''; 72 | 73 | if (languageMatches(language) && sectionMatches()) { 74 | result = separator + src + '\n'; 75 | } 76 | 77 | return result; 78 | }; 79 | 80 | renderer.listitem = function (text) { return text; }; 81 | renderer.list = function (body, ordered) { return body; }; 82 | 83 | marked.use({ renderer: renderer }); 84 | 85 | var output = marked.parse(src); 86 | output = output.replace(/\n+$/g, ''); 87 | 88 | return output.substring(separator.length); 89 | }; 90 | 91 | root.codedown = codedown; 92 | 93 | if (typeof module !== 'undefined' && typeof exports === 'object') { 94 | module.exports = codedown; 95 | } else if (typeof define === 'function' && define.amd) { 96 | define(function() { return codedown; }); 97 | } else { 98 | root.codedown = codedown; 99 | } 100 | })(this || (typeof window !== 'undefined' ? window : global)); 101 | -------------------------------------------------------------------------------- /package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "codedown", 3 | "description": "Extract and run code blocks from Markdown files", 4 | "version": "3.2.1", 5 | "homepage": "https://github.com/earldouglas/codedown", 6 | "author": { 7 | "name": "James Earl Douglas", 8 | "email": "james@earldouglas.com", 9 | "url": "http://earldouglas.com" 10 | }, 11 | "repository": { 12 | "type": "git", 13 | "url": "git://github.com/earldouglas/codedown.git" 14 | }, 15 | "bugs": { 16 | "url": "https://github.com/earldouglas/codedown/issues" 17 | }, 18 | "license": "MIT", 19 | "bin": { 20 | "codedown": "./codedown.js" 21 | }, 22 | "main": "lib/codedown.js", 23 | "engines": { 24 | "node": ">= 0.10.0" 25 | }, 26 | "dependencies": { 27 | "arg": "5.0.2", 28 | "marked": "4.3.0", 29 | "minimatch": "3.1.2", 30 | "readline": "1.3.0" 31 | }, 32 | "keywords": [ 33 | "markdown", 34 | "literate programming" 35 | ], 36 | "devDependencies": { 37 | "coveralls": "3.1.1", 38 | "istanbul": "0.4.5", 39 | "jshint": "2.13.6", 40 | "mocha": "9.2.2", 41 | "mocha-lcov-reporter": "1.3.0" 42 | }, 43 | "jshintConfig": { 44 | "laxcomma": true 45 | }, 46 | "scripts": { 47 | "test": "jshint . --exclude node_modules && mocha", 48 | "coverage": "istanbul cover -no-default-excludes -x '**/node_modules/**' _mocha -- -R spec", 49 | "coveralls": "cat ./coverage/lcov.info | coveralls" 50 | } 51 | } 52 | -------------------------------------------------------------------------------- /test/codedown.js: -------------------------------------------------------------------------------- 1 | var assert = require('assert'); 2 | var process = require('child_process'); 3 | 4 | describe('codedown', function(){ 5 | 6 | it('should require a argument', function (done) { 7 | process.exec('./codedown.js', function (err, stdout, stderr) { 8 | if (!err) { 9 | assert.equal( 10 | [ 'Usage: codedown [...]' 11 | , '' 12 | , 'Options:' 13 | , '--separator ' 14 | , '--section
' 15 | , '' 16 | , 'Example:' 17 | , 'cat README.md | codedown haskell --separator=----- --section 1.3' 18 | , '' 19 | ].join('\n'), 20 | stdout 21 | ); 22 | done(); 23 | } else { 24 | console.log(stderr); 25 | } 26 | }); 27 | }); 28 | 29 | it('should extract code', function (done) { 30 | process.exec('cat README.md | ./codedown.js haskell', function (err, stdout, stderr) { 31 | if (!err) { 32 | assert.equal( 33 | stdout, 34 | [ 'x :: Int' 35 | , 'x = 42' 36 | , '' 37 | , 'main :: IO ()' 38 | , 'main = putStrLn $ show x' 39 | , '' 40 | ].join('\n') 41 | ); 42 | done(); 43 | } else { 44 | console.log(stderr); 45 | } 46 | }); 47 | }); 48 | 49 | it('should extract code with wildcard', function (done) { 50 | process.exec('cat README.md | ./codedown.js "**/*.js"', function (err, stdout, stderr) { 51 | if (!err) { 52 | assert.equal( 53 | stdout, 54 | 'var x = 42;\n' 55 | ); 56 | done(); 57 | } else { 58 | console.log(stderr); 59 | } 60 | }); 61 | }); 62 | 63 | it('should extract code with separator', function (done) { 64 | process.exec('cat README.md | ./codedown.js haskell --separator=-----', function (err, stdout, stderr) { 65 | if (!err) { 66 | assert.equal( 67 | stdout, 68 | [ 'x :: Int' 69 | , 'x = 42' 70 | , '-----' 71 | , 'main :: IO ()' 72 | , 'main = putStrLn $ show x' 73 | , '' 74 | ].join('\n') 75 | ); 76 | done(); 77 | } else { 78 | console.log(stderr); 79 | } 80 | }); 81 | }); 82 | 83 | it('should extract code by section number (1)', function (done) { 84 | process.exec('cat README.md | ./codedown.js haskell --section 1', function (err, stdout, stderr) { 85 | if (!err) { 86 | assert.equal( 87 | stdout, 88 | [ 'x :: Int' 89 | , 'x = 42' 90 | , '' 91 | , 'main :: IO ()' 92 | , 'main = putStrLn $ show x' 93 | , '' 94 | ].join('\n') 95 | ); 96 | done(); 97 | } else { 98 | console.log(stderr); 99 | } 100 | }); 101 | }); 102 | 103 | it('should extract code by section number (1.3)', function (done) { 104 | process.exec('cat README.md | ./codedown.js haskell --section 1.3', function (err, stdout, stderr) { 105 | if (!err) { 106 | assert.equal( 107 | stdout, 108 | [ 'x :: Int' 109 | , 'x = 42' 110 | , '' 111 | , 'main :: IO ()' 112 | , 'main = putStrLn $ show x' 113 | , '' 114 | ].join('\n') 115 | ); 116 | done(); 117 | } else { 118 | console.log(stderr); 119 | } 120 | }); 121 | }); 122 | 123 | it('should extract code by section number (1.3.1)', function (done) { 124 | process.exec('cat README.md | ./codedown.js haskell --section 1.3.1', function (err, stdout, stderr) { 125 | if (!err) { 126 | assert.equal( 127 | stdout, 128 | [ 'x :: Int' 129 | , 'x = 42' 130 | , '' 131 | ].join('\n') 132 | ); 133 | done(); 134 | } else { 135 | console.log(stderr); 136 | } 137 | }); 138 | }); 139 | 140 | it('should extract code by section number (1.3.2)', function (done) { 141 | process.exec('cat README.md | ./codedown.js haskell --section 1.3.2', function (err, stdout, stderr) { 142 | if (!err) { 143 | assert.equal( 144 | stdout, 145 | [ 'main :: IO ()' 146 | , 'main = putStrLn $ show x' 147 | , '' 148 | ].join('\n') 149 | ); 150 | done(); 151 | } else { 152 | console.log(stderr); 153 | } 154 | }); 155 | }); 156 | 157 | it('should extract code by section name (## Examples)', function (done) { 158 | process.exec('cat README.md | ./codedown.js haskell --section "## Examples"', function (err, stdout, stderr) { 159 | if (!err) { 160 | assert.equal( 161 | stdout, 162 | [ 'x :: Int' 163 | , 'x = 42' 164 | , '' 165 | , 'main :: IO ()' 166 | , 'main = putStrLn $ show x' 167 | , '' 168 | ].join('\n') 169 | ); 170 | done(); 171 | } else { 172 | console.log(stderr); 173 | } 174 | }); 175 | }); 176 | 177 | it('should extract code by section name (### Variables in different languages)', function (done) { 178 | process.exec('cat README.md | ./codedown.js haskell --section "### Variables in different languages"', function (err, stdout, stderr) { 179 | if (!err) { 180 | assert.equal( 181 | stdout, 182 | [ 'x :: Int' 183 | , 'x = 42' 184 | , '' 185 | ].join('\n') 186 | ); 187 | done(); 188 | } else { 189 | console.log(stderr); 190 | } 191 | }); 192 | }); 193 | 194 | it('should extract code by section name (### Console output in different languages)', function (done) { 195 | process.exec('cat README.md | ./codedown.js haskell --section "### Console output in different languages"', function (err, stdout, stderr) { 196 | if (!err) { 197 | assert.equal( 198 | stdout, 199 | [ 'main :: IO ()' 200 | , 'main = putStrLn $ show x' 201 | , '' 202 | ].join('\n') 203 | ); 204 | done(); 205 | } else { 206 | console.log(stderr); 207 | } 208 | }); 209 | }); 210 | 211 | it('should extract any code if language is *', function (done) { 212 | process.exec('cat README.md | ./codedown.js "*"', function (err, stdout, stderr) { 213 | if (!err) { 214 | assert.equal( 215 | stdout, 216 | [ 'x :: Int' 217 | , 'x = 42' 218 | , '' 219 | , 'var x = 42;' 220 | , '' 221 | , 'val x = 42' 222 | , '' 223 | , 'main :: IO ()' 224 | , 'main = putStrLn $ show x' 225 | , '' 226 | , 'console.log(x);' 227 | , '' 228 | , 'println(x)' 229 | , '' 230 | ].join('\n') 231 | ); 232 | done(); 233 | } else { 234 | console.log(stderr); 235 | } 236 | }); 237 | }); 238 | 239 | }); 240 | -------------------------------------------------------------------------------- /test/task-list.js: -------------------------------------------------------------------------------- 1 | var codedown = require('../lib/codedown.js'); 2 | var assert = require('assert'); 3 | 4 | describe('task list', function(){ 5 | it('should not leak brackets', function (done) { 6 | var input = 7 | [ 8 | "* [X] checked", 9 | "* [ ] unchecked", 10 | "", 11 | "```lang", 12 | "code", 13 | "```", 14 | ].join('\n'); 15 | var output = codedown(input, "lang"); 16 | assert.equal(output, 'code'); 17 | done(); 18 | }); 19 | }); 20 | --------------------------------------------------------------------------------