UP | HOME

Org Workflow
#org#emacs

Table of Contents

My workflow closely aligns with the philosophy of 'Getting Things Done', yet despite David Allen's insistence on using a pad of paper, I use org mode features to keep myself organized.

A workflow is the sequence of processes through which a project gets done. A project is defined as any goal which requires more than one step to reach.

Ideal workflow

The ideal workflow is sustainable, conspicuous, discretized and flexible.

Sustainability is a core requirement of any workflow. If a workflow is too tedious or adds too much overhead work it will be dropped almost immediately.

Workflows should be extremely conspicuous. Every time you open your computer you should see some part of your workflow and it should be thoroughly reviewed once a week.

Workflows should consist of discrete processes. Filing, for example, should not be confused with reviewing, which should both be distinct from processing work.

Finally, workflows must be flexible enough to fit every type of project. For me, a knitting project, a university class, and flexibility training are all incorporated into one workflow.

Context Switching & Discrete Workflows

One way to improve the distinctiveness of processes in a workflow is by limiting the amount of context switching within each process.

Context switching is a response to your environment changing. Sometimes when I walk through a doorway I forget what I came into the room to get. In the context of workflows and computing, a similar thing may happen when you switch tabs in a web browser or go from filling out a form to trying to find a file to upload. It can even be as simple as switching from your keyboard to the mouse.

Emacs, as a whole, is a great tool to limit context switching. Throughout this article I'll be sharing my configuration options which allow me to singularly focus on each process within the workflow.

File structure

~/org  
  IN.org
  projects.org
  ref.org
  support/

My projects and supporting material are all stored in my-org-directory:

(setq my-org-directory "~/org")

Three org files exist in this folder:

  1. projects.org - A complete list of projects and the steps they contain
  2. ref.org - Reference file often backlinked to projects.org
  3. IN.org - ad-hoc place to capture information which will later be refiled to projects.org and ref.org

Along with these files, a support folder exists to store files like pdfs and images which are often backlinked to ref.org.

Because no projects exist within the support folder, I set my org-agenda-files to be any file directly in my-org-directory, not recursively:

;; Include all files in `my-org-directory' but not recursively
(setq org-agenda-files (list my-org-directory))

;; Include all files in `my-org-directory' and all subfolders
;; (setq org-agenda-files my-org-directory)

Processes

My workflow consists of four distinct processes:

  1. Capturing
  2. Filing
  3. Processing work (or just 'working')
  4. Review

Capturing

workflow-capturing.svg

This is the process where I define a project. An impetous to start a project may come from an email, a web page, a text message, or a conversation. When it happens, I start a new heading with the title of the project as the goal which needs to be accomplished and any supporting info pasted underneath. Capturing can also add supporting info or new steps to an existing project.

All captures end up in IN.org. This is the place for un-processed work and un-filed reference information. Sometimes I write directly in IN.org but, for the most part, I use org capture templates to quickly input information.

Org capture templates can be configured to be launched from virtually any environment. I will show you how I have it set up to be launched from four environments: emacs, dired, a web browser, and gnus.

General settings

;; Launch org capture template selection
(global-set-key (kbd "C-c c") 'org-capture)

;; Align all tags after capturing
(add-hook 'org-capture-mode-hook #'org-align-all-tags)

;; Set capture templates to be an empty list
(setq org-capture-templates '())

From emacs

For a quick note, I can press C-c c n from anywhere in emacs.

capture-emacs.gif

;; Add 'quick note' template
(add-to-list 'org-capture-templates
             `("n" "Quick note" entry
               (file ,(concat my-org-directory "IN.org"))
               "* %^{Title}\n  %U\n\n  %?"
               :prepend t)
             t)

This will prompt for a title in the minibuffer and give you a heading with an inactive timestamp and point placed on the first line of the content.

From dired

Capturing a file into the workflow is a two step process: first you store a link to the file within dired with C-c l, then you capture the link using C-c c l.

capture-dired.gif

(define-key dired-mode-map (kbd "C-c l") 'org-store-link)

(add-to-list 'org-capture-templates
             `("l" "Quick link" entry
               (file ,(concat my-org-directory "IN.org"))
               "* %:annotation\n  %U\n\n  %?"
               :prepend t)
             t)

From the browser

capture-browser.gif

Two bookmarks called 'Org capture' and 'Org capture & edit' are on my bookmarks bar. Each contains a javascript snippet to share webpage metadata directly to IN.org. The first will do so without any interaction, while the second allows you to put more context in before committing. For each, any text selected on the webpage will be pasted into the content of the captured heading.

This solution uses org protocol which relies on emacs' daemon running in the background:

(require 'server)

;; Start server if not running already
(unless (server-running-p)
  (server-start))
  • Require org protocol:
    (require 'org-protocol)
    
  • (linux) Register protocol handler

    Register org-protocol:// handler for xdg-open, allowing a browser to launch emacsclient from a link:

    # /usr/local/share/applications/emacsclient.desktop
    [Desktop Entry]
    Name=Emacs Client
    Exec=/usr/bin/emacsclient %u
    Icon=emacs-icon
    Type=Application
    Terminal=false
    MimeType=x-scheme-handler/org-protocol;
    
  • Bookmarks

    Add two bookmarks to your favorite browser:

    • Org capture

      javascript:location.href='org-protocol://capture?template=Lh&url='+encodeURIComponent(location.href)+'&title='+encodeURIComponent(document.title)+'&body='+encodeURIComponent(window.getSelection())
      
    • Org capture & edit

      javascript:location.href='org-protocol://capture?template=Li&url='+encodeURIComponent(location.href)+'&title='+encodeURIComponent(document.title)+'&body='+encodeURIComponent(window.getSelection())
      
  • Capture templates

    Finally, add the corresponding capture templates 'Link (headless)' and 'Link (interactive)':

    (add-to-list 'org-capture-templates '("L" "Org protocol link") t)
    
    (add-to-list 'org-capture-templates
                 `("Lh" "Link (headless)" entry
                   (file ,(concat my-org-directory "IN.org"))
                   "* %:annotation\n  %U\n\n  %i"
                   :prepend t
                   :immediate-finish t
                   :kill-buffer t)
                 t)
    
    (add-to-list 'org-capture-templates
                 `("Li" "Link (interactive)" entry
                   (file ,(concat my-org-directory "IN.org"))
                   "* %:annotation\n  %U\n\n  %i\n\n  %?"
                   :prepend t)
                 t)
    

From Gnus

I use Gnus for email and rss feeds, both of which can be captured to org. Each captured message is moved to a different group in Gnus (to prevent archiving) before a link is created. This is all done in one step using the backslash (\) key. If a message is already in the correct group, C-c c will simply capture it without moving.

capture-gnus.gif

  • Capture templates

    For capturing email, a temporary list of capture templates specific to gnus messages is used instead of org-capture-templates.

    (setq gnus-capture-templates
          `(("f" "Follow up" entry
             (file ,(concat my-org-directory "IN.org"))
             "* TODO Follow up with %:fromname on %:subject :email:%:group:
      %a
    
      %i"
             :prepend t)
            ("l" "Later to read" entry
             (file ,(concat my-org-directory "IN.org"))
             "* TODO Read %:subject :email:%:group:
      %a
    
      %i"
             :prepend t)
            ("r" "Reference" entry
             (file ,(concat my-org-directory "IN.org"))
             "* %^{Headline} :email:%:group:
      %a
    
      %i"
             :prepend t)))
    

    Just like for capturing from the browser, any text selected will be pasted into the content of the captured heading. f (Follow up) is used if I need to reply to a message later, l (Later to read) is used if I need to read a message but I'm not sure what to do with it afterwards, and r (Reference) is used when I want to save the message as a link in my reference file.

  • Gnus move and capture

    This is the lisp function which does the work of moving and capturing a gnus message.

    (require 'ol)
    (require 'ol-gnus)
    (require 'org-capture)
    
    ;; Rename gnus links to use new group name after moving a message
    (defun org-gnus-store-link-replace-group (link)
      "If `new-group' is defined, replace it in stored LINK."
      (if (boundp 'new-group)
          (let ((new-link (replace-regexp-in-string "\\(^gnus:\\)[^#]*\\(#.*\\)"
                                                    (format "\\1%s\\2" new-group)
                                                    link)))
            (org-link-add-props :link new-link
                                :group new-group)
            new-link)
        link))
    
    (advice-add 'org-gnus-store-link :filter-return
                #'org-gnus-store-link-replace-group)
    
    
    (defun gnus-capture (&optional goto keys)
      "Capture the current email.
    
    Prefix arguments are passed to `org-capture'."
      (interactive "P")
      (let ((org-capture-templates gnus-capture-templates))
        (org-store-link nil t)
        (org-capture goto keys)))
    
    (define-key gnus-summary-mode-map (kbd "C-c c") #'gnus-capture)
    (define-key gnus-article-mode-map (kbd "C-c c") #'gnus-capture)
    
    
    (defun gnus-move-and-capture (&optional goto keys)
      "Move email to a new group then capture it.
    
    Prefix arguments are passed to `org-capture'."
      (interactive "P")
      (let ((new-group (gnus-read-move-group-name
                        "move" gnus-current-move-group
                        (gnus-summary-work-articles 1) nil))
            (org-capture-templates gnus-capture-templates))
        (gnus-summary-move-article nil new-group)
        (org-store-link nil t)
        (org-capture goto keys)))
    
    (define-key gnus-summary-mode-map (kbd "\\") #'gnus-move-and-capture)
    (define-key gnus-article-mode-map (kbd "\\") #'gnus-move-and-capture)
    

Filing

workflow-filing.svg

At least once a day, I go through my IN.org file and move each item to where it belongs. Everyone's reference and projects file will be organized somewhat uniquely and organically as you add more items and review it every week. In order to refile a heading, use C-c C-w and type in the file and outline path to the required destination.

;; Set refile targets to all headings in the current buffer and all
;; headings in the agenda files.  If
;; `org-outline-path-complete-in-steps' is nil, you may want to lower
;; the ':maxlevel' setting to limit the amount of refile targets.
(setq org-refile-targets '((nil :maxlevel . 9)
                           (org-agenda-files :maxlevel . 9)))

;; Allow refile as top level heading to other files
(setq org-refile-use-outline-path 'file)

;; Show all refile targets (at all depths) immediately
;; (setq org-outline-path-complete-in-steps nil)

;; Refile in steps, choosing the file first, then parent
;; headline and so on
(setq org-outline-path-complete-in-steps t)

;; Ability to create new parent node while refiling
(setq org-refile-allow-creating-parent-nodes 'confirm)

During this process, I can also contextualize each item using keywords, tags, effort, schedules or deadlines.

Keywords

A keyword marks the status of a step in a project. To set one on a heading, use C-c C-t.

(setq org-todo-keywords
      '((sequence
         "TODO(t!)" ;; Default keyword. Log timestamp
                    ;; automatically. Used for tasks that are not
                    ;; immediate or that have a specific date, like an
                    ;; appointment. Can be scheduled or deadlined.

         "NEXT(n!)" ;; The very next thing to do for a specific
                    ;; project. Log timestamp automatically. Should
                    ;; only be one per project.

         "WAITING(w@)" ;; Waiting on an external event, like a package
                       ;; to arrive or work to be done by another
                       ;; person. Log a note when entering this state
                       ;; describing who or what you are waiting on.

         "SOMEDAY(s)" ;; A task without a schedule or deadline.
         "|"
         "DONE(d!)" ;; Completed task. Log timestamp automatically.

         "CANCELLED(c@)"))) ;; Cancelled task. Log a note for why this
                            ;; task was cancelled.

All steps in a project are TODOs, but only one is a NEXT. TODOs are promoted to NEXT as the project is completed. I don't always hold myself to the rule that each project should have only one NEXT; if two tasks can be completed simoultaneously and are both equally important they can both be NEXT.

Tags

Tags denote the type of the work required to complete a task split into two groups: context and energy. Context is used to define the physical requirements of a task, like needing to be online, in the city, or on your phone. Energy is the mental state needed to excell at a task like 'mindful', or 'social'. From each group, a heading may have at most one tag.

You can set tags for a heading using C-c C-q

(setq org-tag-alist '((:startgroup)
                      ("context")
                      (:grouptags)
                      ("web" . ?w)
                      ("phone" . ?p)
                      ("outdoors" . ?o)
                      ("city" . ?c)
                      ("dev" . ?d)
                      (:endgroup)
                      (:startgroup)
                      ("energy")
                      (:grouptags)
                      ("mindful" . ?m)
                      ("tedious" . ?t)
                      ("social" . ?s)
                      ("physical" . ?P)
                      (:endgroup)))

Effort

The approximate amount of time it will take to complete a task. This can be set for a heading using C-c e.

(add-to-list 'org-global-properties '("Effort_ALL" . "0:05 0:15 2:00 4:00"))

Schedules and Deadlines

Some work has a deadline or schedule that you must keep, like filing taxes or a doctors appointment. Items that must occur at a specific time in the future can be SCHEDULED with C-c C-s and items that must occur by a certain time in the future can have a DEADLINE with C-c C-d. When rescheduling an item, make sure to log a note as to why it was rescheduled.

(setq org-log-reschedule 'note
      org-log-redeadline 'note)

Processing Work

workflow-working.svg

The rest of the workflow can be considered 'overhead', while this is the meat and potatoes. Ideally, more time should be spent processing work than on any other process in the workflow. But the big question is what you should be doing with your time.

A day's work can be split into three categories: required, ad-hoc, and undirected. Required work is any task with a SCHEDULED or DEADLINE cookie. Ad-hoc work comes to you in the moment and takes your attention away from required or undirected work. These types of projects can live and die in IN.org, or, if done immediately, may not even be tracked at all. Finally, undirected work is when you are in control of what project is being worked on at any moment.

The first thing to look at when deciding what to do should always be the calendar. Like Allen, I believe the schedule is a sacred place and should be given respect. Only put things that absolutely must happen on the schedule, and never miss any TODO's with a SCHEDULED or DEADLINE cookie. Tasks like 'clean the fishtank' should not be scheduled, to do so would cheapen the importance of something like 'tickets to Wharton Center'. And, overscheduling and falling behind on tasks will be depressing and inhibit uptake of the entire workflow.

To view this week's calendar, use C-c a a. Obviously, any task due today should be done today, but getting ahead on other tasks, like preparing for an upcoming presentation, is also a good idea.

Ad-hoc work doesn't need a special key sequence. In fact, it's quite hard to ignore ad-hoc work like 'unplug the toilet'. Nevertheless, it's still a good idea to check IN.org a couple times a day for things you may be forgetting.

Undirected work is whenever you have a choice on what to work on. I can look at all tasks with C-c a t, tasks marked NEXT with C-c a T NEXT, and tasks matching a tag query like C-c a m tedious+web to match any task tagged tedious and web, or C-c a m social|phone to match any task that is tagged social or phone.

Conclusion

I've been working on this workflow for more than a year now and it has served me well. Being able to see an overview of all of my projects is very rewarding and calming to me. I'm always trying to track new things using this system and work on ways to make it more visible and less obtrusive.