Caching Emacs Binaries in GitHub Workflows
1. Background
This blog is built as a static site using the Emacs orgmode publishing system1 (see publish/publish.el). I'll probably write about the details eventually. For the purposes of this post, my main requirements were:
- All posts are written in orgmode
- Local preview is easy with the standard org export dispatcher
- Deployments run in GitHub Actions and fast–under one minute
My main concern when bootstrapping the build was (3). Orgmode's publish system requires Emacs (and possibly some external packages), which is usually a major version or two behind in Linux package managers and takes several minutes to build/install from source.
Fortunately, purcell/setup-emacs exists and does all the heavy lifting. A
drop-in workflow step uses nix
to install pre-built binaries in about 50
seconds per job.
The challenge was to get that down to 5 seconds or less, leaving plenty of time in a minute for runners to boot, build the site, and sync to S3.
2. Setup
The approach I landed on was to cache emacs and its dependencies in the GitHub
action cache. I was new to nix
, but after only a few minutes of reading about
the project, I had a good feeling. Indeed, nix
provides all the tools to make
this easy.
Four steps must be added to the workflow. First, check the GitHub cache. I hardcode a path in the runner's
$HOME
and generate a key based on the OS (always
Linux here) and Emacs major/minor version. This step restores the cache to the specified path.
- name: Cache emacs - Get id: cache-emacs uses: actions/cache@v3 with: path: /home/runner/.local/nix/store-emacs-29-1 key: ${{ runner.os }}-emacs-29-1
Second, if we don't have emacs binaries cached, install with setup-emacs
. This step just
conditionally invokes the action from the marketplace, depending on the cache
step result, installing emacs to the nix
store in the runner.
- name: Cache emacs - Miss - Install uses: purcell/setup-emacs@master if: steps.cache-emacs.outputs.cache-hit != 'true' with: version: 29.1
Third, also on a cache miss, save the emacs binary and all its dependencies to the cache path. nix-store --query --requisites
lists all the files we need.
actions/cache
will automatically update the cache at the end of the job.
- name: Cache emacs - Miss - Put if: steps.cache-emacs.outputs.cache-hit != 'true' run: | ls -l `which emacs` emacs --version mkdir -p $HOME/.local/nix/store-emacs-29-1 nix-store --query --requisites $(which emacs) | xargs -I {} rsync -av {} $HOME/.local/nix/store-emacs-29-1
Fourth, on a cache hit, simply link the restored cache binaries into /nix/store
where they expect to live and add
emacs to the runner's PATH
.
- name: Cache emacs - Hit if: steps.cache-emacs.outputs.cache-hit == 'true' run: | sudo mkdir -p /nix/store sudo ln -s $HOME/.local/nix/store-emacs-29-1/* /nix/store/ emacs_store=$(find /nix/store -name "*-emacs-29-1") test -x $emacs_store/bin/emacs echo "$emacs_store/bin" >> $GITHUB_PATH
That's it: two marketplace actions and two steps with a couple bash commands.
3. Closing Thoughts
Caching is easy to implement for a single job and works as intended. Loading emacs from the cache takes 2 seconds per build.
It is a bit manual and repetitive for multiple jobs, though. Ideally, setup-emacs
would manage
the cache internally using the GitHub action tool cache. I think that
would require a rewrite of the action from bash to JS/TS, though. For this job, the manual
setup works perfectly for me.
Footnotes
For now. Orgmode's publishing system has some nice features around linking within the project and sitemap generation. However, there are pain points. Templating HTML in Lisp is not fun. Generating a feed of all project files by chaining through the generated sitemap feels cumbersome and limiting. I can already imagine moving to a simpler pandoc-based build down the line.