September 16, 2023

Caching Emacs Binaries in GitHub Workflows

1. Background

This blog is built as a static site using the Emacs orgmode publishing system1 (see publish/publish.el). I'll probably write about the details eventually. For the purposes of this post, my main requirements were:

  1. All posts are written in orgmode
  2. Local preview is easy with the standard org export dispatcher
  3. Deployments run in GitHub Actions and fast–under one minute

My main concern when bootstrapping the build was (3). Orgmode's publish system requires Emacs (and possibly some external packages), which is usually a major version or two behind in Linux package managers and takes several minutes to build/install from source.

Fortunately, purcell/setup-emacs exists and does all the heavy lifting. A drop-in workflow step uses nix to install pre-built binaries in about 50 seconds per job.

The challenge was to get that down to 5 seconds or less, leaving plenty of time in a minute for runners to boot, build the site, and sync to S3.

2. Setup

The approach I landed on was to cache emacs and its dependencies in the GitHub action cache. I was new to nix, but after only a few minutes of reading about the project, I had a good feeling. Indeed, nix provides all the tools to make this easy.

Four steps must be added to the workflow. First, check the GitHub cache. I hardcode a path in the runner's $HOME and generate a key based on the OS (always Linux here) and Emacs major/minor version. This step restores the cache to the specified path.

- name: Cache emacs - Get
  id: cache-emacs
  uses: actions/cache@v3
  with:
    path: /home/runner/.local/nix/store-emacs-29-1
    key: ${{ runner.os }}-emacs-29-1

Second, if we don't have emacs binaries cached, install with setup-emacs. This step just conditionally invokes the action from the marketplace, depending on the cache step result, installing emacs to the nix store in the runner.

- name: Cache emacs - Miss - Install
  uses: purcell/setup-emacs@master
  if: steps.cache-emacs.outputs.cache-hit != 'true'
  with:
    version: 29.1

Third, also on a cache miss, save the emacs binary and all its dependencies to the cache path. nix-store --query --requisites lists all the files we need. actions/cache will automatically update the cache at the end of the job.

- name: Cache emacs - Miss - Put
  if: steps.cache-emacs.outputs.cache-hit != 'true'
  run: |
    ls -l `which emacs`
    emacs --version
    mkdir -p $HOME/.local/nix/store-emacs-29-1
    nix-store --query --requisites $(which emacs) | xargs -I {} rsync -av {} $HOME/.local/nix/store-emacs-29-1

Fourth, on a cache hit, simply link the restored cache binaries into /nix/store where they expect to live and add emacs to the runner's PATH.

- name: Cache emacs - Hit
  if: steps.cache-emacs.outputs.cache-hit == 'true'
  run: |
    sudo mkdir -p /nix/store
    sudo ln -s $HOME/.local/nix/store-emacs-29-1/* /nix/store/
    emacs_store=$(find /nix/store -name "*-emacs-29-1")
    test -x $emacs_store/bin/emacs
    echo "$emacs_store/bin" >> $GITHUB_PATH

That's it: two marketplace actions and two steps with a couple bash commands.

3. Closing Thoughts

Caching is easy to implement for a single job and works as intended. Loading emacs from the cache takes 2 seconds per build.

It is a bit manual and repetitive for multiple jobs, though. Ideally, setup-emacs would manage the cache internally using the GitHub action tool cache. I think that would require a rewrite of the action from bash to JS/TS, though. For this job, the manual setup works perfectly for me.

Footnotes

1

For now. Orgmode's publishing system has some nice features around linking within the project and sitemap generation. However, there are pain points. Templating HTML in Lisp is not fun. Generating a feed of all project files by chaining through the generated sitemap feels cumbersome and limiting. I can already imagine moving to a simpler pandoc-based build down the line.


RSS

Creative Commons License This work is licensed under a Creative Commons Attribution 4.0 International License.