UP | HOME

How this blog works
#org#syncthing

Table of Contents

This blog is a creative outlet for me that allows me to refine my understanding of the topics I'm interested in. I don't even have any tracking mechanisms because I judge each article's value by my own set of standards rather than external appreciation.

That being said, I would love to hear how to improve the site. I'm thinking of adding a comment section or an email list option in the near future and I will be sure to update this post when I do.

UPDATE July 30, 2021: I've added tags.

UPDATE Aug 6, 2021: I've added comments.

UPDATE Mar 3, 2022: I've added tracking.

General process

I write in org mode and export to html. The css is written afterwards instead of modifying the export process. This allows me to use the chrome inspector to quickly optimize an element rather than going through a build-and-test cycle. The element attributes ox adds to each html element are descriptive and unique enough to target quite easily with css selectors.

I also use the org-info-js script to add keybindings, advanced TOC options, and section folding to the website.

Dark mode is enabled using dark-mode-toggle and css media queries.

The org files are synced to my server using Syncthing and I serve the website using emacs.

Folder structure

As I mentioned, the blog is a subset of all my org files which are synced using Syncthing.

         
~/org       My org directory (`my-org-directory')
  /blog      
    ox-config.el   Org export configuration
    /public   The 'source code' for the website
      /index.org Site listing
      /emacs  
      /thoughts My subfolders of different interests
      /web  
      /js Javascript files
      /css CSS Files
      /feed.org RSS Feed source
      /rss.xml RSS Feed

Git vs Syncthing

Git is a great tool for keeping track of plaintext files. It's usefullness with regard to a personal blog is probably overstated. If you find yourself committing messages like 'added some stuff', 'updated blog', or just 'updates', maybe it's time you asked, why bother?

I have a dual boot system as I need to use some Windows applications for work. Whenever switching systems, with git I would need to push and pull all changes each time.

Syncthing is a much more behind-the-scenes approach. It runs at startup and syncs files as they are created. I have it connected to my phone, both operating systems, and a hosted syncthing server in the cloud. If any three of those systems went down, I would still have a full copy of all the files to restore from (it's happened once).

One thing I will say in favor of git is that branches are a really nice tool for graduated deployments. You can have a dev branch and a main branch and only do development on the dev branch and merge into main when you feel comfortable. Programming a website 'live' like I'm doing is definitely not recommended for a serious project.

Local development

In each folder in my blog, there is an index.org file which lists the articles in that folder. When working on a new blog post, the last thing I do is add it to the index.org so that people will not see unfinished work. This is not a security measure in that I would never put sensitive information anywhere in the blog folder; even places that are conventionally inaccessible.

The syncing process takes some time, so I like to run a local web server to get instant feedback. I built org-ssr for this purpose. From the public folder, I run C-u M-x org-ssr which will generate a random port and serve all files recursively in any format I want.

The website is also available on my local network so I can look at it from my phone using my computer's IP address instead of localhost. Most web traffic today is mobile!

Elisp configuration

First, some package setup 

(package-initialize)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
(unless package-archive-contents
  (package-refresh-contents))
(if (package-installed-p 'htmlize)
    (require 'htmlize)
  (package-install 'htmlize))
(require 'ox)
(require 'ox-org)
(require 'htmlize)

Appearance

Here are some appearance-related org export configurations: 

(setq org-export-with-section-numbers nil
      org-export-with-toc nil)

By default, I turn off section numbers and the table of contents.

Syntax highlighting

Syntax highlighting within src blocks is accomplished using the package htmlize 

(use-package htmlize)
(setq org-src-fontify-natively t)

Navigation

(setq org-html-link-home "/"
      org-html-link-up ".")

These options tell the browser to look for an index.html file within the current directory when navigating 'up' and /index.html when navigating 'home'. The forward slash tells the browser to look in the ~/org/web/public folder.

org-info-js

The default org-info-js options look, to me, meant to mirror as closely as possible emacs 'info manuals'. I want something a little more like a classic website so I go with these defaults and configure them a little within each file: 

(setq org-html-use-infojs t
      org-html-infojs-options
      '((path . "/js/org-info.js")
        (view . "showall")
        (toc . "0")
        (ftoc . "0")
        (tdepth . "max")
        (sdepth . "max")
        (mouse . "underline")
        (buttons . "nil")
        (ltoc . "0")
        (up . :html-link-up)
        (home . :html-link-home)))

The first option, the path, is quite important. I've downloaded the js file into ~org/web/public/js/org-info.js and the leading slash tells the browser to look at that folder no matter how deep down within subfolders the user currently is.

All content is shown by default with (view . "showall") because not everyone prefers to use keys to navigate a website. This will let any typical web user to just scroll through the entire article without having to navigate or expand subsections.

Publishing options

(setq my-org-directory "/home/tyler/org") ;; an essential variable in my config

(setq org-publish-project-alist
      `(("blog"
         :base-directory ,(concat my-org-directory "web/")
         :base-extension "org"
         :publishing-directory ,(concat my-org-directory "web/public/")
         :publishing-function org-html-publish-to-html
         :recursive t)))

This allows me to publish all articles in the web folder at once with M-x org-publish-project RET blog RET. If I ever change any of the elisp mentioned here, I will have to run this in order to update each web page.

In order to publish a single article, I run C-c C-e h h from within an org file.

Custom css/js/html

I've already mentioned that I prefer to edit the appearance of the website using an external css file. I also added the dark-mode-toggle script as an external file. Here's how that looks in elisp: 

(setq org-html-head "
      <link rel=\"stylesheet\" type=\"text/css\" href=\"/css/main.css\"/>
        <link rel=\"stylesheet\"
              type=\"text/css\"
              href=\"/css/dark.css\"
                media=\"(prefers-color-scheme: dark)\">
        <script type=\"module\" src=\"/js/dark-mode-toggle.mjs\"></script>")

This snippet references three files

  • ~/org/web/public/css/main.css
  • ~/org/web/public/css/dark.css
  • ~/org/web/public/js/dark-mode-toggle.mjs

The first two are manually created and the last can be downloaded from unpkg. The main.css holds imports to all other css files and the dark.css is only applied when 'dark mode' is requested by the user (or if their system preferences specify to use dark mode).

Dark mode

Now that I have the dark mode script installed, I will place the actual toggle button in the preamble of the org export. 

(setq org-html-preamble-format '(("en" "<dark-mode-toggle
        appearance=\"switch\"
        dark=\"Dark\"
        light=\"Light   \"
        remember=\"Remember\"></dark-mode-toggle>")))

Tags

A wonderful reddit user by the name of IntelligentTea281 asked me how I manage tags on this blog. The answer is I didn't even think about it!

So now I do this:

I have a tags.org file in the ~/org/web directory that looks like this: 

#+TITLE: Tags
#+EXPORT_FILE_NAME: public/tags
#+HTML_LINK_UP: /
#+EXCLUDE_TAGS: ignore
#+CALL: createTagFile()
#+INFOJS_OPT: view:info sdepth:1

* Tag file creation                                                  :ignore:
  #+name: createTagFile
  #+BEGIN_SRC emacs-lisp :results value raw
    (defun org-article-get-descriptor-if-filetag (tag file)
      "If org file has a FILETAG matching tag, get the file path and
    title as a list."
      (with-temp-buffer
        (insert-file-contents file)
        (let ((properties (org-export-get-environment)))
          (if (seq-contains-p (plist-get properties :filetags) tag)
              (list (file-relative-name file)
                    (substring-no-properties
                     (car (plist-get properties :title))))))))

    (defun org-articles-matching-filetag (tag files)
      "Get a list of org articles which have a FILETAG matching `tag'"
      (sort 
       (seq-filter
        (lambda (a) (not (null a)))
        (mapcar
         (lambda (file)
           (org-article-get-descriptor-if-filetag tag file))
         files))
       (lambda (a b) (string-lessp (car (cdr a)) (car (cdr b))))))

    (defun org-html-create-tag-listing (tag files)
      "Create an org ast of links to articles which contain a tag"
      (let ((articles (org-articles-matching-filetag tag files)))
        (unless (null articles))
        `((headline (:title "Tags" :level 1))
          (property-drawer () (node-property (:key "CUSTOM_ID" :value ,tag)))
          (headline (:title ,(format "#%s" tag) :level 2))
          ,(mapcar (lambda (article)
                     `(headline (:title ,(apply 'format "[[file:%s][%s]]" article)
                                        :level 3)))
                   articles))))

    (defun org-html-create-tags-listing (dir)
      "Create an org-mode document string for listing tags from all
    files in `dir'"
      (let* ((files (directory-files-recursively
                     dir "^\\([^.#]\\)\.+.org$"))
             (tags (sort (mapcar
                          'car
                          (org-global-tags-completion-table files))
                         (lambda (a b) (string-lessp a b))))
             (ast (mapcar
                   (lambda (tag)
                     (org-html-create-tag-listing tag files))
                   tags)))
        (org-element-interpret-data ast)))


    (org-html-create-tags-listing (concat my-org-directory "web/"))
  #+END_SRC

When exporting this file, the code block will run and insert the results directly into the buffer. Because of the #+EXCLUDE_TAGS property, the actual source block won't be shown, only the results of evaluation.

What `org-export-create-tag-lists' does is look through a folder recursively for org tags. For each tag in that list, it looks again recursively and builds a list of articles that have a #+FILETAGS property that includes that tag. It reads the relative filename and title of the file and formats it as an org link. Also, it creates a custom ID for the heading that can be linked to from each article.

I've made it so each tag is its own section by using the #+INFOJS_OPT: view:info sdepth:1 property. That way, when linking to a specific tag, only articles matching that tag are shown.

There's one more step though, and that is actually showing the #+FILETAGS to the user on each article. That is done with some elisp advice: 

(defun org-export-add-filetags-to-subtitle (args) 
  "Include filetags as the subtitle in an org export"
  (let ((info (car (cdr args)))) 
    (plist-put
     info 
     :subtitle `(export-block 
                 (:type "HTML" 
                  :value ,(mapconcat
                           (lambda (tag) 
                             (format
                              "<span class=\"tag\"><span class=\"%s\"><a href=\"/tags#%s\">#%s</a></span></span>"
                              tag tag tag)) 
                           (plist-get info :filetags)
                           ""))))
    args))

(advice-add 'org-html-template 
            :filter-args #'org-export-add-filetags-to-subtitle)

This will add a subtitle on an article with the list of #+FILETAGS during export. Each tag will link to /tags#<tag>. I don't typically use subtitles so this works out well for me.

I also add a link inside each regular tag by overwriting the org-html--tags function. Regular tags are not used to build the tag lists so these must match existing #+FILETAGS. 

(defun org-html--tags (tags info) 
  "Format TAGS into HTML.
  INFO is a plist containing export options."
  (when tags
    (format
     "<span class=\"tag\">%s</span>"
     (mapconcat
      (lambda (tag) 
        (format
         "<span class=\"%s\"><a href=\"/tags#%s\">#%s</a></span>"
         (concat (plist-get info :html-tag-class-prefix) 
                 (org-html-fix-class-name tag))
         tag tag))
      tags
      ""))))

Thanks for the thought provoking comment, IntelligentTea281!

RSS Feed

To enable RSS export of an org file, install the org-contrib package and load ox-rss: 

(if (not (package-installed-p 'org-contrib))
    (package-install 'org-contrib))
(require 'ox-rss)

Create a file named feed.org in the public folder which will be exported to an rss.xml file.

In order to publish an article to the rss feed, create a new top-level headline in feed.org which will be the title of the RSS entry. Add the property RSS_PERMALINK to point to the relative webpage the entry should link to using C-c C-x p RSS_PERMALINK. 

#+TITLE: Tyler Grinn
#+AUTHOR: Tyler Grinn
#+EXPORT_FILE_NAME: rss

* Simple Nav Bar
  :PROPERTIES:
  :RSS_PERMALINK: web/nav-bar
  :END:

  Create a simple html nav bar and collapsible menu

Export the document as an rss file using C-c C-e r r. This will add an :ID: and :PUBDATE: property to each top-level heading and create rss.xml with all headings listed there. Be sure to save the org file after exporting.

Disqus

I use disqus to display a comment box on each article. Disqus automatically stores and moderates the comments for me and allows quick reactions.

First, each article needs a unique ID. I create this by using M-x org-id-get-create and copying the generated id to an #+ID: file property at the top of the file.

In order to use this ID, I have to add it to the format spec when exporting by advising the org-html-format-spec function. 

(defun org-html-add-id-to-format-spec (orig info)
  (let (spec
        (id (with-temp-buffer
              (insert-file-contents (plist-get info :input-file))
              (cadr (car (org-collect-keywords '("ID")))))))
    (setq spec (funcall orig info))
    (add-to-list 'spec `(?i . ,id))))

(advice-add 'org-html-format-spec :around #'org-html-add-id-to-format-spec)

This adds %i as a valid replacement string in the preamble and postamble for getting the unique ID of a file.

I added disqus to the postamble using a modified version of the disqus universal script. This checks whether the ID is currently available and loads the comments. You'll need to replace 'https://blog.tygr.info' with the base location of your blog. By hardcoding the origin, local development will not accidentally create a new comment thread. Also, replace 'https://tyler-grinn.discuss.com/embed.js' with the correct url given to you by discus. 

(setq org-html-postamble t)
(setq org-html-postamble-format
      '(("en" "
        <div id=\"disqus_thread\"></div>
        <script>
         var disqus_config = function () {
           this.page.url = \"https://blog.tygr.info\" + location.pathname;
           this.page.identifier = \"%i\";
         };

         function loadComments() {
           var d = document, s = d.createElement('script');
           s.src = 'https://tyler-grinn.disqus.com/embed.js';
           s.setAttribute('data-timestamp', +new Date());
           (d.head || d.body).appendChild(s);
         };

         if (\"%i\" !== 'nil') loadComments()
        </script>
        <p class=\"footer\">%a &nbsp; | &nbsp; %C</p>
       ")))

Notice the page identifier is being set to \"%i\". Now every file that has an "ID" property will also have a comments section.

The actual footer is the last line and will look something like this:

Tyler Grinn | 2021-07-20 Tue 08:36

Latex configuration

In order to support latex export of my org files, I need to install texlive with apt install texlive. Afterwards, latex and tlmgr should be on the path. 

(require 'ox-latex)

Syntax highlighting

To colorize org babel src code blocks, I'll use the minted package from texlive (tlmgr install minted) and the pygments package from python (apt install python3-pygments). Afterwards, pygmentize should be on the path. 

;; Add code syntax highlighting
(setq org-latex-listings 'minted)
(add-to-list 'org-latex-packages-alist
             '("cache=false,newfloat,outputdir=/tmp" "minted"))
(setq org-latex-minted-options '(("breaklines" "true")
                                 ("breakanywhere" "true")
                                 ("linenos" "true")))

The outputdir=/tmp is necessary because pygments creates .pyg files in the output directory which minted needs to know about, and asynchronously exporting org files will create the exported latex file in the /tmp folder.

breaklines and breakanywhere tell minted how to handle the case where lines are longer than the page width. With line breaks, it's useful to turn line numbering on with linenos to help the reader understand what's happening.

Setting the graphics path and converting gif files

I will override the #+LATEX_HEADER_EXTRA property for all org files in order to accomplish two goals: set the graphicspath to fix broken inline image links and convert the first frame of every gif file into a png for use in the resulting pdf file. 

;; for converting gif to png
(add-to-list 'org-latex-packages-alist
             '("" "epstopdf"))

;; Set LATEX_HEADER_EXTRA for all org files
(advice-add 'org-latex-make-preamble :filter-args
            #'(lambda (args)
                (plist-put (car args) :latex-header-extra (concat "
% Set path to look for graphics files
\\graphicspath{{" (file-name-directory (plist-get (car args) :input-file)) "}}

% Convert gif (first frame) to png
\\epstopdfDeclareGraphicsRule{.gif}{png}{.png}{convert \\SourceFile[0] \\OutputFile}

% Add gif as a valid inline image extension
\\AppendGraphicsExtensions{.gif}"))
                args))

;; Org export latex should recognize 'gif' as an inline-able image
(setq org-latex-inline-image-rules
      `(("file" . ,(rx "."
                       (or "gif" "pdf" "jpeg" "jpg" "png" "ps" "eps" "tikz" "pgf" "svg")
                       eos))))

Org files

Each of my org blog files start with four lines: 

#+TITLE: How this blog works
#+ID: l529oft056j0
#+EXPORT_FILE_NAME: ../public/emacs/blog
#+DESCRIPTION: All the elisp, js, and css behind the scenes

Depending on how big an article is, I may add some tables of contents: 

#+INFOJS_OPT: toc:t tdepth:1 ltoc:above

This line in particular (which applies to this article) gives me multiple tables of contents: one at the top and one in each section that has subheaders. The top table of contents is limited to showing only the top-level headers with tdepth:1.

If I want to add some tags to an article, I use #+FILETAGS 

#+FILETAGS: :org:syncthing:kubernetes:

Beyond that, the org file is just an org file. Tables, src blocks, quotes are all exported beautifully without any special consideration.

As I mentioned, each folder has an index.org file which looks something like this: 

# -*- org-html-use-infojs: nil; -*-
#+TITLE: Tyler Grinn | Emacs
#+EXPORT_FILE_NAME: ../public/emacs/index
#+HTML_LINK_UP: /

* Emacs articles
** How this blog works
** Custom mode line
   Detailed instructions on how to customize your mode line without
   slowing down emacs
** I have strong opinions on the shape of your cursor

I disable infojs for the index files because having collapsable headings makes it tricky to click if they are also a link.

The HTML_LINK_UP option is necessary because the default I set tells the browser to go to the index.html file, which is already what is being shown. So this option should navigate the user to the index.html file above the current subdirectory. The only thing 'above' the emacs folder is the root directory of the website, so I put / for the 'up' action.

Notice the lack of an ID property means that index pages will not have a comment box.

The links are all relative 'file' links. Org automatically renames these from .org links to .html when exporting.

CSS files

You can find whole books about how to organize css code. Putting it all in one file is certainly a possibility, but I would urge a little separation of concerns. My current philosophy is to split the css into four areas of consideration:

  • Layout
  • Fonts
  • Theme
  • Components

As such, the main.css file is simply 

@import 'layout.css';
@import 'fonts.css';
@import 'theme.css';
@import 'components.css';

The first three affect the website globally. The layout is for gross layout considerations, basically how to position the different boxes. Fonts and themes are self-explanatory. The components file is where I modify the default look of individual org elements.

The css is where you can really make your blog unique. Use SVGBackgrounds.com to add a cool pattern, add some fancy fonts, and read up on CSS-Tricks in order to put your own flavor on the blog.

I'm a simple person, and as a simple person my theme.css file is currently sitting empty. Oh well…

You can look at the each of my files in the browser in order to get an idea of how to write your own.

Conclusion

Blogging is a new experience for me. I always check articles thrice before publishing and still feel nervous promoting it anywhere. This website is primarily for myself but I'd still love any feedback.

Tyler Grinn   |   2022-05-16 Mon 22:53