My first package: eventuel
#emacs#node#gitlab

In order to package eventuel in the cloud for each commit using Eldev, I added a build stage using silex/emacs as the base image:

package:
  stage: build
  image: silex/emacs:27
  before_script:
    - curl -fsSL https://raw.github.com/doublep/eldev/master/webinstall/eldev | sh
  script:
    - /root/.eldev/bin/eldev -dtT lint
    - /root/.eldev/bin/eldev -dtT package
  artifacts:
    paths:
      - dist/

The command eldev package creates a dist folder which is uploaded to Gitlab's artifact repository. In the dist folder is a packed archive from a fresh clone that can be installed using M-x package-install-file.

I wanted to use the Gitlab Release api for hosting tagged versions of the package. In order to do this from the CI/CD template, I had to specify the direct url to the artifact.

release:
  stage: release
  only:
    - tags
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  dependencies:
    - package
  release:
    tag_name: $CI_COMMIT_TAG
    description: $CI_COMMIT_DESCRIPTION
    assets:
      links:
        - name: eventuel-$CI_COMMIT_TAG.tar
          url: $CI_PROJECT_URL/-/jobs/$CI_JOB_ID/artifacts/raw/dist/eventuel-$CI_COMMIT_TAG.tar
  script:
    - echo Release
  artifacts:
    paths:
      - dist/
    expire_in: never

Now, when I tag the repository (eg git tag 0.1.0 && git push origin 0.1.0), the artifacts from the package job are re-downloaded into the release job and archived once again, except this time with the flag expire_in: never to make sure Gitlab doesn't clean up this release accidentally.

The link is a direct-download link to the artifact dist/eventuel-0.1.0.tar, which will be shown on the Releases page of the repository.

Finally, the commit description (everything after the first line) is set as the release notes. I haven't figured out how to format this properly, it may be rendered as markdown.

It was tricky to figure out how to use the filesets options in Eldev, but finally I figured out how to translate the complex 'composite' fileset eldev-standard-excludes into a simple list. From there I could ignore Node depedencies and let the package install them from the web after installation.

(setq eldev-standard-excludes '(".*" "node_modules/" "/dist/"))

I still can't figure out how to tell npm to only install devDependencies for the linting stage in the CI/CD template, so instead I split the package.json up into two files: package.json and server/package.json. The former is only for linting while the latter is only for runtime dependencies.

I figured out a pretty elegant way to set the prefix for a global command keymap from a user-defined option.

The solution requires the easy-mmode package:

(require 'easy-mmode)

First I defined the custom prefix as nil to start:

(defcustom eventuel-prefix nil
  "Prefix key for the eventuel keymap."
  :type 'string
  :group 'eventuel)

I defined the mode map to be an empty sparse keymap:

(defvar eventuel-mode-map (make-sparse-keymap)
  "Keymap for managing nodes and users on the eventuel server.")

I defined the command map using easy-mmode-define-keymap:

(defvar eventuel-command-map
  (easy-mmode-define-keymap
   (mapcar (lambda (k) (cons (kbd (car k)) (cdr k)))
           '(("n n" . eventuel-get-nodes)
             ("n c" . eventuel-create-node)
             ("n d" . eventuel-delete-node)
             ("u u" . eventuel-get-users)
             ("u c" . eventuel-create-user)
             ("u d" . eventuel-delete-user)
             ("u p" . eventuel-set-password))))
  "Postfix keymap to be used in `eventuel-mode-map'.")

Finally I defined the minor mode with the options :keymap eventuel-mode-map. Within the startup function, I check for a user-defined prefix and modify the eventuel-mode-map. It's important not to reassign the variable or else the changes won't take effect.

(when eventuel-prefix
  (setcdr eventuel-mode-map nil) ;; Remove previous prefix bindings
  (define-key eventuel-mode-map (kbd eventuel-prefix) eventuel-command-map))

I made extensive use of the request library for interacting asynchronously with the REST server. I defined three utility functions which made the rest of the code a lot easier on the eyes.

Sqlite3 is a very basic implementation of sqlite without any async functions or the tagged template literal function that I'm used to. I defined a utility function sql to do both of these things at once:

const sqlite3 = require('sqlite3');

const { DB_FILENAME } = process.env;

const db = new sqlite3.Database(DB_FILENAME);

exports.default = (strings, ...args) =>
  new Promise((resolve, reject) => {
    db.all(strings.join('?'), args, (err, rows) => {
      if (err) reject(err);
      else resolve(rows);
    });
  });

Now I can just await sql expressions without worrying about matching up args and question marks or endless nesting callback functions.

const sql = require('./sql').default;

async function getSomeStuff(isAwesome = true) {
  const stuff = await sql`
    SELECT stuff
    FROM myTable
    WHERE awesome=${+isAwesome}
  `;
  return stuff;
}

Because async functions are simply a wrapper around returning a promise, it's equivalent to simply do this:

const sql = require('./sql').default;

const getSomeStuff = (isAwesome = true) => sql`
  SELECT stuff
  FROM myTable
  WHERE awesome=${+isAwesome}
`;

I use an SSE stream in order to send events to the web client. On both the client and the server, the setup was a lot simpler than I thought and required no external packages. For the expressjs side, I defined a handler that takes a callback function for sending data:

exports.default = (handler) => (req, res) => {
  res.writeHead(200, {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache',
    Connection: 'keep-alive',
  });

  handler(
    req,
    {
      send: (data) => res.write(`data: ${JSON.stringify(data)}\n\n`),
      onClose: (cb) => res.on('close', cb),
    },
  );
};

Usage:

const EventEmitter = require('events');
const { Router } = require('express');

const sse = require('./sse').default;

const logEmitter = new EventEmitter();
const router = Router();

router.get('/stream', sse((_req, conn) => {
  const listener = (entry) => conn.send(entry);
  logEmitter.on('entry', listener);
  conn.onClose(() => logEmitter.off('entry', listener));
}));

router.post('/log', (req, res) => {
  logEmitter.emit(req.body);
  return res.sendStatus(200);
});

On the client side, it's even simpler:

const logStream = new EventSource('/stream');
logStream.onmessage = (event) => {
  console.log('Received log', JSON.parse(event.data));
}

However, one bug was found for kubernetes, where my nginx ingress has a connection timeout separate and out of the control of expressjs. I had to add an annotation to the ingress in order to keep the connection open for long periods of time:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tygr-eventuel
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "86400"