UP | HOME

Org SSR
#emacs#blog#org

Table of Contents

An on-demand org exporting web server.

Motivation

I want org files to be available in multiple formats: .org, .html, .txt, and .pdf, but I don't want to keep track of four files for every article I write. If I update an article, that export process could take several seconds and several keystrokes. Things in my environment, like themes, might change the output accidentally. A web server that serves org files directly overcomes all of that by it's ability to run as an independent lisp machine with only the bare minimum config necessary to export org files.

The web server will export org files in the format they are requested on-demand, which speeds up launch time. The exported files are stored in a cache, so subsequent requests for the same file and filetype will be faster. The cache is invalidated based on the modified timestamp of the file, so updating the blog is as simple as saving a file.

The blog folder itself will be much tidier because exported files will be served either from memory or the /tmp folder.

Finally, users will also be able to use the website from org-mode inside emacs by using browse-url-emacs with a .org page, like M-x browse-url-emacs https://blog.tygr.info/emacs/cursor.org RET

Installation

If not using the docker image, you must install the emacs package. Releases can be found on the github releases page. Download the tar file and use M-x package-install-file to install it. Do the same in order to update.

How to use it

There are three main ways to use org-ssr:

  1. As a docker image
  2. As an interactive command from emacs
  3. As a shell script for systemd

Docker image

A docker image is useful if you want to execute arbitrary babel commands during export and want to isolate the export process, or if you want to run the server in a kubernetes environment.

docker run \
       -p 8080:8080 \
       -e RECURSIVE=t \
       -v /path/to/serve:/public \
       -v /path/to/ox-config.el:/ox-config.el \
       registry.gitlab.com/tygrdev/org-ssr:latest

Now, the web server will be available on port 8080 and serve /path/to/serve recursively using /path/to/ox-config.el as the org-export-async-init-file.

org-ssr Command

The org-ssr command will serve all files in the current directory, but not recursively. Use the C-u universal prefix argument to enable serving subdirectories. Two prefix arguments (C-u C-u M-x org-ssr) will allow other computers on the local network access the server, such as your phone, and finally three prefix arguments (C-u C-u C-u M-x org-ssr) will serve the current folder recursively and be available on the local network.

The serve script

org-ssr ships with a serve script located at ~/.emacs.d/elpa/org-ssr-x.y.z/serve.

This script can be used in a systemd service configuration to automatically start on boot and restart in case of a failure:

[Unit]
Description=Blog

[Service]
ExecStart=/home/tyler/.emacs.d/elpa/org-ssr-1.0.0/serve
Restart=on-failure
Environment=OX_CONFIG=/home/tyler/org/blog/ox-config.el
Environment=DIR=/home/tyler/org/blog/public
Environment=RECURSIVE=t
Environment=PORT=8080
Environment=BABEL=t

[Install]
WantedBy=default.target

Special considerations for latex

Latex export can be a little tricky when done asynchronously. Three problems I ran into were:

  1. Babel src blocks were not colored
  2. Linked images not showing up
  3. Gif images shown as broken links

Latex export of babel src blocks

I'm using the latex package minted which requires the python package pygments in order to use its pygmentize command during exporting. If using texlive, you can install minted using tlmgr install minted and you can install python pygments with apt install python3-pygments (or possibly apt install python-pygments).

In order for the latex compiler to be able to execute arbitrary commands during export, the --shell-escape flag must be added to org-latex-pdf-process. If using the systemd script, this is as simple as setting the environment variable SHELL_ESCAPE to t. Otherwise, you can add this to your emacs config:

(setq org-latex-pdf-process
 '("latexmk -f -pdf -%latex --shell-escape -interaction=nonstopmode -output-directory=%o %f"))

This is a very dangerous setting and should only be applied if you are sure no malicious code exists in any exported latex file.

In the ox-config.el script used for asynchronous exporting, the minted package needs to be set up:

(setq org-latex-listings 'minted)
(add-to-list 'org-latex-packages-alist
             '("cache=false,newfloat,outputdir=/tmp" "minted"))
(setq org-latex-minted-options '(("breaklines" "true")
                                 ("breakanywhere" "true")
                                 ("linenos" "true")))

The option outputdir=/tmp is necessary because the org async export process puts the resulting latex code in the /tmp folder and minted needs to know where it is.

Setting the graphics path

Again, because the asynchronous export process places the latex file in /tmp, any inline image with a relative path will be broken in the latex export. This can be fixed by setting an absolute path for the \graphicspath{{}} command in the #+LATEX_HEADER of your org file:

#+LATEX_HEADER: \graphicspath{{/home/tyler/org/blog/public/emacs}}

This can be done automatically for all org files by overriding the LATEX_HEADER_EXTRA property during export. Fair warning, this means that setting #+LATEX_HEADER_EXTRA on any org file will not have any effect.

;; ox-config.el

;; Set LATEX_HEADER_EXTRA for all org files
(advice-add 'org-latex-make-preamble :filter-args
            #'(lambda (args)
                (plist-put (car args) :latex-header-extra (concat "
% Set path to look for graphics files
\\graphicspath{{" (file-name-directory (plist-get (car args) :input-file)) "}}"))
                args))

This will always set the graphicspath to be the absolute directory of the org file, so relative links will work as expected.

Converting gif files to png

PDF files cannot have moving images, so when asking for an export, any gif file should have its first frame converted to a png.

First, I need to tell ox-latex to include gif files as inline images by modifying org-latex-inline-image-rules

(setq org-latex-inline-image-rules
      `(("file" . ,(rx "."
                       (or "gif" "pdf" "jpeg" "jpg" "png" "ps" "eps" "tikz" "pgf" "svg")
                       eos))))

I will include the built-in latex package epstopdf for converting images:

;; for converting gif to png
(add-to-list 'org-latex-packages-alist
             '("" "epstopdf"))

I will modify the latex-header-extra override I defined above to include a new graphics rule for gif files and register .gif as a valid graphics extension:

;; Set LATEX_HEADER_EXTRA for all org files
(advice-add 'org-latex-make-preamble :filter-args
            #'(lambda (args)
                (plist-put (car args) :latex-header-extra (concat "
% Set path to look for graphics files
\\graphicspath{{" (file-name-directory (plist-get (car args) :input-file)) "}}

% Convert gif (first frame) to png
\\epstopdfDeclareGraphicsRule{.gif}{png}{.png}{convert \\SourceFile[0] \\OutputFile}

% Add gif as a valid inline image extension
\\AppendGraphicsExtensions{.gif}"))
                args))

Again, because this uses the shell command convert, shell escape must be enabled for it to work.

Query parameters

With the options :no-cache t and :babel t, query parameters can be used in babel source code blocks in order to change the appearance of a web page based on query parameters passed in by the user.

:babel t is required to execute babel source code blocks on export and :no-cache t is required to prevent 'lock in' of the first request's query parameters for future requests.

For example, to enable pagination, you can declare a var using the (org-ssr-query) function:

#+header: :var page=(org-ssr-query :page 0 'number)
#+begin_src emacs-lisp :exports results :results value raw
  (format "* Page number %d" page)
#+end_src

#+RESULTS:
* Page number 0

Now, when someone requests ?page=2, then the babel code block will produce:

* Page number 2