Interactive Writing: A plea for better communication
3 |
A few years ago, I first read the excellent essay by Bret Victor, "What can a technologist do about climate change?." For its treatment of climate change alone, I can't recommend the essay enough—there's food for thought to keep you satiated for a few months. But, then, near the end, Victor sneaks in a little section titled "Model-driven debate" that has has kept me thinking for years.
4 |
He begins with the example of Alan Blinder's "Cash for Clunkers" proposal. The federal government would offer car owners a rebate to exchange old, inefficient vehicles for newer ones. Proponents claimed it would cause massive emissions reductions. Meanwhile, critics claimed there were more cost-effective ways to reduce emissions. Who's right?
5 |
Of course, it's both and neither—the answer depends entirely on the parameters of the program. As Victor writes:
6 |
7 |
"Many claims made during the debate offered no numbers to back them up. Claims with numbers rarely provided context to interpret those numbers. And never were readers shown the calculations behind any numbers. Readers had to make up their minds on the basis of hand-waving, rhetoric, bombast."
8 |
9 |
Victor asks us to imagine a better world: what if the author had proposed a model rather than mere words? Then, we, the readers, could make up our own minds. Instead of bombast, we get an informed debate about the underlying assumptions and resulting tradeoffs.
10 |
Let's look at an example (a slight modification of Victor's original example1):
11 |
12 |
Say we allocate for the following program: Car-owners who trade in an old car that gets less than , and purchase a new car that gets better than , will receive a rebate.
13 |
We estimate that this will get off the road. It will save of gas (or worth of U.S. gas consumption.) It will avoid CO2e, or % of annual U.S. greenhouse gas emissions.
14 |
The abatement cost is per ton CO2e of federal spending, although it’s per ton CO2e on balance if you account for the money saved by consumers buying less gas.
15 |
16 |
Try sliding clicking and dragging the items in green to update their values. You'll see the items in blue change as a result. To see how these outputs are computed, click on one of the blue items, and you'll see the calculation in the appendix to this article.
17 |
18 | When I first saw this example, I had the kind of feeling that I imagine people in the '80s must have had when they first saw wheels on a suitcase, that of dockworkers when they first encountered shipping containers in the 60s, or of late 15th century Europeans when they first read the results of movable type. A combination of "oh that's
19 | so obvious!" with the shame of your civilization not having come up with the idea earlier and something akin to disgust at how we used to do things (or are still doing them).
20 |
21 |
Victor's vision is what journalism and argumentative writing should look like. Next to this better system, hand-waving opinion pieces border on offensive.
22 |
23 | Unfortunately, his vision has gotten almost no attention since its conception. Victor provided a small library, Tangle, to implement models like these, but not much has happened with it in the last half decade. That's understandable—the library requires prior experience with web development, which makes it unapproachable for most people, but it also offers no direct integration with any major JavaScript (JS) framework, which does not encourage actual web
24 | developers to use it.
25 |
26 |
27 | In its place, we've seen success with somewhat similar projects like Observable. Observable helps you write JS notebooks that are highly interactive and relatively easy to embed in other websites. But the experience is not seamless: you still need familiarity with JS. Of course, we've had Jupyter notebooks and R Markdown for a while. Unfortunately all of these notebook-based models remain somewhat clunky and cumbersome. None of them
28 | offer a really fluent and easy inline input option like Tangle.
29 |
30 |
In this post, I'd like to look at a middleground—a (almost) no-code way to create interactive documents, which offers a much easier writing experience at the cost of sacrificing some of the customizability of Tangle or Observable/Python/R notebooks. Let's call it interactive Markdown.
31 |
Now, I'm not the first. Shortly after Victor published Tangle, there was an explosion in Markdown related integrations: dynamic Markdown, active Markdownfangle, and TangleDown are what I could find. I'm sure there are yet more.
32 |
Still, I think there's a good reason to reinvent this wheel. For one, I'm not happy about the syntax of any of these options (though least unhappy with that of active/dynamic Markdown). The problem is that none of them are backwards compatible with existing Markdown interpreters. I'm of the strong opinion that since there are so many Markdown extensions already, if you come up with a new, it had better be backwards compatible.
33 |
Second, all but fangle miss the ability to do inline calculations. Third, none is actively maintained. Fourth, all of them work by compiling .md to .html; I'd like an option to compile to .jsx from .mdx, which I think would generally make this much easier to adopt for other people. Five, none offer an elegant way to display supplementary calculations the way Victor's example did.
34 |
There's also a good "cultural" reason to reinvent this wheel. Thanks to note-taking tools like Notion, Roam, and Obsidian, Markdown is having a moment. More people are playing around with Markdown than ever before, so if ever there were a time to build on Markdown, it's now.
35 |
Without further do, let me present interactive Markdown.
36 |
37 |
An example
38 |
Let's take a look at a very simple example (again from Victor):
39 |
40 |
When you eat , you consume . That's of your recommended daily calories.
41 |
42 |
Under the hood, this looks as follows:
43 |
When you eat [3 cookies](cookies=[0..100]),
44 | you consume **[150 calories](calories=50*cookies)**.
45 | That's [7.5%](daily_percent) of your recommended daily calories.
46 |
47 |
Interactive Markdown is built around "fields". There are three in this example: [3 cookies](cookies=[0..100]), [150 calories](calories=50*cookies), and [7.5%](daily_percent).
48 |
If you're familiar with Markdown, then you'll recognize a field as a link. Like a link, every field is made up of two parts ([text representation](variable configuration)): a text representation of the element between square brackets [](the link text or alt text for a media element) and the variable configuration between round brackets ()(the link href or image src).
49 |
The reason for using the same syntax as a link is backwards compatibility. If there is no interactive Markdown interpreter, you only lose interactivity, not the reading experience.
50 |
There are three kinds of fields: input, output, and reference fields.
51 |
Input fields
52 |
[3 cookies](cookies=[0..100]) is an input field. In the variable configuration, (cookies=[0..100]), we define a variable, cookies, that takes its value from a range of 0 to 100. In the text representation, [3 cookies], we give the default value, 3. The surrounding text is used as a template (for example, to specify units).2
53 |
There are two kinds of input field, range and select:
54 |
55 |
Range Input (my_var=[min..max;step]): By clicking on the range input and dragging left or right, the user can adjust its value between min and max in intervals of size step.
56 |
Select Input (my_var=[a,b,c]): By clicking on the select input, the user cycles through the options a, b, c...
57 |
58 |
Output fields
59 |
[150 calories](calories=50*cookies) is an output field. On the right, we define the variable calories as the product of 50 and our previously defined variable cookies.
60 |
Since the definition contains neither a range [min..max;step] nor select [a,b,c] input, an output field is not directly adjustable via user input. It is dynamically computed from the other variables in a document's scope.
61 |
Because of this, the value of 150 is really more like a fallback than a default. An interactive Markdown interpreter won't ever user this value. A standard Markdown interpreter will render it as 150 calories for the same experience just without the interactive part.
62 |
Reference fields
63 |
Lastly, [7.5%](daily_percent) is a reference field. Unlike definition fields (i.e., input and output fields) references do not contain an equal sign = in their variable configuration. They display a variable that has already (or will be) defined elsewhere in the page.
64 |
For example, we might put the calculation for daily_percent in the appendix to avoid cluttering the body text for your reader:
65 |
66 |
Calculation for daily_percent
67 |
68 |
Daily recommended calories limit =
69 |
Percent cookie calories per day =
70 |
71 |
72 |
Behind the scenes, this is:
73 |
### Calculation for `daily_percent`
74 | - Daily recommended calories limit = [2,000 calories](daily_calories=[0..5000;50])
75 | - Percent cookie calories per day = [7.5%](daily_percent=calories/daily_calories)
76 |
77 |
78 |
References are useful for separating long calculations from your story line. It also helps to remind readers what variable values are, so they don't have to scroll back and forth a hundred times.
79 |
Each variable should only have one definition field but can have arbitrarily many reference fields.
80 |
Note that reference fields act differently depending on whether they reference an input or output variable:
81 |
82 |
Input references let you update the original variable. To the reader, input references are indistinguishable from input definitions.
83 |
Output references link to the original output definition. So I recommend you define an output variable in the same place that you describe its calculation to readers.
84 |
85 |
86 |
Conclusion
87 |
For the time being, it will take some technical know-how to get interactive Markdown up and running for yourself. If you're interested, I've written a remark plugin that you can drop into an existing remark/rehype pipeline.
88 |
That's because interactive Markdown is still in its infancy. There are many features I'd like to get to that I haven't had the time for yet (e.g., automatic dimensions checking to make sure your calculations make sense, popover links to calculations, more math functions, support for distributions and other data types), not to mention tools to make working with interactive Markdown easier: an in-browser editor, a plugin for Obsidian support, etc.
89 |
If you're interested in all of this, make sure to subscribe to my newsletter to stay updated. And if you have any ideas, I'd love to hear from you. Check out the repository and raise an issue (or, even better, send a pull request).
90 |
91 |
Appendix
92 |
A more complicated example
93 |
Let's look at the more complicated example from the beginning.
94 |
Here is the example again (thanks to reference fields, it's perfectly in sync with the first instance):
95 |
96 |
Say we allocate for the following program: Car-owners who trade in an old car that gets less than , and purchase a new car that gets better than , will receive a rebate.
97 |
We estimate that this will get off the road. It will save of gas (or worth of U.S. gas consumption.) It will avoid CO2e, or % of annual U.S. greenhouse gas emissions.
98 |
The abatement cost is per ton CO2e of federal spending, although it’s per ton CO2e on balance if you account for the money saved by consumers buying less gas.
99 |
100 |
And here's what it actually looks like (the first example):
101 |
Say we allocate [$3.0 billion](budget=[0..10;0.1]&margin-right=1ch) for the following program:
102 | Car-owners who trade in an old car that gets less than [17 MPG](old_MPG_limit=[5..30]),
103 | and purchase a new car that gets better than [24 MPG](new_MPG_limit=[5..50]),
104 | will receive a [$3,500](rebate=[0..20000;100]&margin-right=1ch) rebate.
105 |
106 | We estimate that this will get [828,571 old cars](cars_traded&margin-right=1ch) off the road.
107 | It will save [1,068 million gallons](gallons_saved&margin-right=1ch) of gas
108 | (or [68 hours](hours_of_gas&margin-right=1ch&margin-right=1ch) worth of U.S. gas consumption).
109 | It will avoid [9.97 million tons](tons_CO2_saved&margin-right=1ch) CO2e,
110 | or [0.14](_percent_annual_emissions)% of annual U.S. greenhouse gas emissions.
111 |
112 | The abatement cost is [$301](dollars_per_ton_CO2e&margin-right=1ch) per ton CO2e of federal spending,
113 | although it’s [-\$20](dollars_per_ton_CO2e_on_balance&margin-right=1ch) per ton CO2e on balance
114 | if you account for the money saved by consumers buying less gas.
115 |
116 |
117 |
A few points to note:
118 |
119 |
The number in the text representation determines display precision. If you're familiar with format strings, 3.0 is converted to %.1f, 17 to %d, 3,500 to %'d3, etc..
120 |
121 |
You can also use format strings directly in the text representation, e.g., [%'d old cars](cars traded), but I don't recommend this because it won't be compatible with standard Markdown.
122 |
123 |
124 |
[0..10;0.1] specifies a range with a step-size equal to 0.1. By default, the step size is 1.
125 |
I haven't figured out spacing yet (hence &margin-right=1ch)
126 |
127 |
Cars Traded
128 |
129 |
budget =
130 |
overhead =
131 |
rebate =
132 |
cars_traded = (budget - overhead) / rebate =
133 |
134 |
Here you see one more trick in interactive Markdown: A link containing an inline code element of the kind [`variable_name`](variable_name) is a reference label. It gets a TKLabel class for easier formatting, and, eventually, will synchronously darken whenever you highlight any references to or dependencies of its variable.
135 |
136 |
Gallons Saved
137 |
This is where my example diverges from Victor's example. His calculation uses the distribution of mileage over current cars and cars being sold. I haven't yet added distributions to the interactive Markdown spec (though I plan to), so you'll have to accept a less precise version. Note that the comments come from Victor's original work.
138 |
Average mileage of old vehicles
139 |
140 |
Assume that traded-in cars are chosen with equal probability from the pool of eligible cars. We use the harmonic average because we'll be calculating gallons consumed for constant miles, so we really want to be averaging gallons-per-mile.
141 |
142 |
143 |
old_MPG_limit =
144 |
average_current_MPG =
145 |
var_current_MPG =
146 |
average_old_MPG = ∫ x N(x; average_current_MPG, var_current_MPG) from -Infinity to old_MPG_limit =
147 |
148 |
Alright so I haven't even actually added support for more complicated formulas like this. But it is coming.
149 |
Average mileage for vehicles currently being sold
150 |
151 |
Assume that new cars are purchased with equal probability from the pool of eligible cars. The distribution really should be sales-weighted. I'm sure the data is available, but I couldn't find it.
152 |
153 |
154 |
new_MPG_limit =
155 |
average_new_MPG =
156 |
var_new_MPG =
157 |
average_new_MPG = ∫ x N(x; average_new_MPG, var_new_MPG) from new_MPG_limit to Infinity =
158 |
159 |
Average gallons saved per car replaced
160 |
161 |
Assume that everyone who is buying a new car now would have eventually bought a similar car when their current car got too old. So the fuel savings from the program should be calculated over the remaining lifetime of the old car. Ideally we'd like the joint distribution of MPGs and age of the current fleet, but I can't find that data. So we'll just use averages.
The importance of models may need to be underscored in this age of “big data” and “data mining”. Data, no matter how big, can only tell you what happened in the past. Unless you’re a historian, you actually care about the future — what will happen, what could happen, what would happen if you did this or that. Exploring these questions will always require models. Let’s get over “big data” — it’s time for “big modeling”.
CO2 comprises 95% of a car's greenhouse gas emissions. The other 5% include methane, nitrous oxide, and hydroflourocarbons. To account for these other gases, we divide the amount of CO2 by 0.95 to get CO2e (“carbon dioxide equivalent”).1
The difference is that I haven't yet added the possibility of inputting a distribution. So the calculations for average MPG of old versus new cars is less precise than in Victor's case. (On the flip side, this coarser model is easier to modify for today's transportation fleet.) ↩↩2
355 |
356 |
357 |
It's a little confusing that cookies shows up on both the left- and right-hand sides. On the right-hand side, it has a semantic purpose: defining the variable cookies. On the left-hand side it has a purely stylistic purpose (to inform the reader what units we're using). ↩
358 |
359 |
360 |
Note: %'d is actually nonstandard. It puts commas (or periods) in the thousands places (depending on your locale). Another useful nonstandard addition is + or - for optionally separating the amount and magnitude as in -$20. ↩