Author’s Note: I’m currently in the process of migrating old blog posts to this new system. That may mean some links, syntax highlighting, and other details are broken or missing temporarily. Sorry for the inconvenience!
I love Markdown. I use it to write these blog posts, I use it to write notes. I use it in READMEs for various software projects. I *really* love how expressive you can be in what's otherwise a plain text file. I can easily represent emphasis, quotes, headers, links, images, code snippets, and if all goes wrong, I can fall back to HTML. ## But something is missing. There's one thing that Markdown does not have support for, and it really bothers me: there's no simple convention for comments!
Let’s start with the problem
Let’s imagine you’re writing a blog post about the fierce-backed wildebeast of the antarctic tundra. As you’re writing, you realize you’re not sure if Antarctic should be capitalized (note: it should). What do you do? Do you stop writing and go look it up? Maybe. But that breaks your flow. Better is to just leave a note to come back and check it later. We need to make a quick note, and the way we do that should meet the following criteria:
- It should be fast to write, like
- It should let us write general comments amongst the text just like (Antarctica is great) parenthetical expressions
- It should optionally let us annotate specific parts of the text, like we can do with links, e.g.
I love [Antarctica](https://www.wikiwand.com/en/Antarctica)
- It should be easy for us to search for in our document
- It should be automatically excluded from the published version
Let’s look at what exists already in Markdown.
What does Markdown have?
A few options come to mind:
Write “TODO” or “FIXME”, or as journalists are so fond of, “TK”
Using placeholder text makes it easy (usually) for us to find items that need to be fixed, but it doesn’t allow us to keep notes coupled to the content. Sometimes we may want notes to exist even once the post is published. Push one of these things live too early, and the post could be strewn with
TODO: Stop using placeholders
Markdown doesn’t technically have footnotes, but a lot of Markdown processors do. Here’s what the syntax looks like:
The fierce-backed wildebeast of the antarctic[^1] tundra is really fierce. [^1]: Should this be capitalized?
We could write some code to extract the footnotes on publication, but the content of the footnote is decoupled from its position. And the footnote still doesn’t precisely mark what’s being marked up. Worst of all, this isn’t standard Markdown, so if we relied on this behavior, it might change and then things would break.
Use HTML comments:
<!-- Should antarctica be capitalized? Not sure... -->
Welp, it’s stable, at least. It won’t show up in production (though it will if you click “view source”). Still no way to mark ranges, and ugh typing those comment braces is uncomfortable.
Okay, now that we’ve looked at some of the available solutions, let’s create our own.
Rolling our own annotations
Here’s my proposed syntax:
The fierce-backed wildebeast of the [antarctic][~ should this be capitalized?] tundra is really fierce.[~ This comment doesn't refer to a specific part of the text]
Stealing from the
[^1] footnote syntax, but with slightly different usage. Easy to search for
[~ in a document, lets us mark up specific parts of the text, not too difficult to type or hard to read.
We set up a templating system a while back, so let’s create a version 0.4.0 that automatically strips out that pattern for us. To do that, we just need to update the line that converts the body of our posts from Markdown to HTML:
# Old v0.3.0 code: $(cat $FILENAME | sed '1,/---/d' | kramdown --syntax-highlighter rouge --no-hard-wrap -i GFM) # New v0.4.0 code: $(cat $FILENAME | sed '1,/---/d' | sed 's/\(\[\([^]]*\)\]\)\?\[~\([^]]*\)]/\2/g' | kramdown --syntax-highlighter rouge --no-hard-wrap -i GFM)
That’s a rather unwieldy regular expression we’ve added, but it does our conversion for us:
echo '[antarctica][~ is cool]' | sed 's/\(\[\([^]]*\)\]\)\?\[~\([^]]*\)]/\2/g' # => antarctica echo 'It works[~ obviously] | sed 's/\(\[\([^]]*\)\]\)\?\[~\([^]]*\)]/\2/g' # => It works
Now, unfortunately, we’re doing some rather naïve parsing here, and that means if I use our new template on this post, it’ll strip out all of my
[~ examples! By keeping our logic relatively simple, we lose all nuance. We could write something that replaces
[~ ...] only outside of a code block, but that could start to get complex very quickly.
Showing the annotations for review
One last thing: sometimes I like to be able to see the annotations in the HTML document. Let’s rig up something like that, using this nice little CSS tooltip code I found on the internet!
So now we have a different transformation that needs to happen, and we can extend our
sed command to do it for us:
echo '[antarctica][~ is cool]' | sed 's/\(\[\([^]]*\)\]\)\?\[~\([^]]*\)]/<a data-tooltip="\3" href="#">_\2_<\/a>/g' # => <a data-tooltip=" is cool" href="#">_antarctica_</a> echo 'It works[~ obviously] | sed 's/\(\[\([^]]*\)\]\)\?\[~\([^]]*\)]/<a data-tooltip="\3" href="#">_\2_<\/a>/g' # => <a data-tooltip="" href="#">__</a>
Note that we’re being lazy here and outputing
__ when there’s no match. I’m not sure if there’s a way to set a default for a capture group with
sed, but hey, this is for drafts anyway.
We’ll drop the tooltip CSS in, and update out script to use our extended replacement only if we find
notes: true in the front matter.
#!/bin/bash FILENAME=$1 FRONT_MATTER=$(cat $FILENAME | sed -n '/^---/,/^---/p') NOTES=$(echo "$FRONT_MATTER" | grep notes: | sed 's/.*: //') # ...snip echo " <article id='main-article'> $( if [ "$NOTES" == "true" ]; then cat $FILENAME | sed '1,/---/d' | sed 's/\(\[\([^]]*\)\]\)\?\[~\([^]]*\)]/<a data-tooltip="\3" href="#">_\2_<\/a>/g' | kramdown --syntax-highlighter rouge --no-hard-wrap -i GFM else cat $FILENAME | sed '1,/---/d' | sed 's/\(\[\([^]]*\)\]\)\?\[~\([^]]*\)]/\2/g' | kramdown --syntax-highlighter rouge --no-hard-wrap -i GFM fi ) </article> "
This is another case where we should probably tidy things up a bit, but I got frustrated trying to assign the replacement pattern to a variable (probably something to do with escaping strings), and it works, so we’ll leave things there for now!
Since we’re not using the new template on this post (because it keeps eating my
[~’s), let’s create another post to test with:
--- version: 0.4.0 title: Showing Notes description: A short demo page to show notes lead-image: fountain-pen.jpg url: https://cheerskevin.com/showing-notes.html notes: true --- Showing Notes ============= This is a [brief][~ At least, let's hope it's brief] example of how notes will now work! [~ I hope this is a good test case!]
The joy of plain text
I started this post talking about how much I love Markdown, and really, that applies to plain text files in general. They're so simple and malleable, and we can extend them to do whatever we want.[~ is this perhaps overstatement?] Today we've taken advantage of our ability to do some pre-processing on the text to add something that I truly think is missing from the syntax, all with simple [command-line tools][~ not sure that simple is really correct here. Regexp aren't simple]. There are probably a bunch of ways to manage this more cleanly, but for now, this is going to work for me, and I'm looking forward to being able to annotate things as I write. Cheers!