Deployment

Parsers are dependencies. Add the languages your application highlights to mix.exs, and mix release carries them the way it carries any dependency's assets:

defp deps do
  [
    {:lumis, "~> 0.9"},
    {:lumis_wasm_bundle_web, "~> 0.1"},
    {:lumis_wasm_elixir, "~> 0.26.0"},
    {:lumis_wasm_markdown, "~> 0.26.0"}
  ]
end

Nothing else to prepare. There is no image stage that downloads parsers and no network at boot or at render: the bytes are in the release, one directory per package.

lib/lumis_wasm_elixir-0.26.3/priv/parsers/
lib/lumis_wasm_markdown-0.26.2/priv/parsers/

lumis_wasm_bundle_web has no directory of its own. A bundle carries no parser bytes, only dependencies, so it adds one more of these per member: css, diff, html, javascript, json, tsx and typescript.

Lumis.Languages.load("haskell")
#=> {:error, :not_installed}

Include the languages a document can inject, not only the ones it names. Markdown fences reach whatever language they label, HTML reaches css and javascript, and Elixir reaches comment. A language you missed costs that block its colors and nothing else: Lumis.highlight/2 returns {:ok, html} either way, and logs a warning naming the dependency to add. Check staging logs for it after a deploy; a colorless block is easy to miss on the page.

Docker

The Dockerfile mix phx.gen.release --docker generates needs no changes. mix deps.get fetches the parsers and mix release packages them, so the build and runner stages you already have carry them.

To keep the compiled-module cache on a writable path, set:

ENV LUMIS_DATA_DIR="/app/lumis"

Warm-up

Loading a parser still costs a WASM compile the first time. Move it off the first request with Lumis.Languages.async_load/1 from your application's start/2:

def start(_type, _args) do
  Lumis.Languages.async_load(~w(markdown elixir javascript rust css html comment))

  Supervisor.start_link(children(), strategy: :one_for_one, name: MyApp.Supervisor)
end

It returns immediately, and the result is deliberately not matched on: a warm-up should not be able to stop an application from starting. Failures are logged and highlighting still loads on demand, so the worst case is paying the compile on the first render after all.

Use Lumis.Languages.load/1 when you do want to wait, such as a release task or a smoke test that should fail if a parser is missing.

The compiled-module cache

config :lumis, :data_dir does not decide where parsers come from. It decides where wasmtime keeps compiled modules:

config :lumis, data_dir: "/app/lumis"   # or LUMIS_DATA_DIR

It defaults to the lumis application's own priv/, which a release owns, and is created on first write. Point it somewhere writable and persistent, and a restart skips recompiling. Lose it and the first render of each language is slower; no request fails. On a read-only filesystem it is never written.

Build with Nix

A sandboxed Nix build cannot download the precompiled Lumis NIF while mixRelease compiles dependencies. The failure may be reported as Error while downloading precompiled NIF: erofs or eacces because rustler_precompiled cannot create its cache under the builder's home directory.

Download the NIF while fetchMixDeps has fixed-output network access, keep the archive in that derivation, and point the release build at the cached copy:

let
  pname = "my_app";
  version = "0.1.0";
  src = ./.;

  mixDeps = beamPackages.fetchMixDeps {
    pname = "mix-deps-${pname}";
    inherit src version;
    hash = "sha256-...";
    mixEnv = "prod";

    postInstall = ''
      export RUSTLER_PRECOMPILED_GLOBAL_CACHE_PATH="$out/.rustler-precompiled"
      mix deps.compile nimble_options --no-deps-check
      mix deps.compile rustler_precompiled --no-deps-check
      mix deps.compile lumis --no-deps-check
      rm -f "$RUSTLER_PRECOMPILED_GLOBAL_CACHE_PATH"/metadata-*.exs
    '';
  };
in
beamPackages.mixRelease {
  inherit pname src version;
  mixFodDeps = mixDeps;
  mixEnv = "prod";

  preConfigure = ''
    export RUSTLER_PRECOMPILED_GLOBAL_CACHE_PATH="$MIX_DEPS_PATH/.rustler-precompiled"
  '';
}

The NIF archive becomes part of the mixFodDeps hash. That hash is now specific to the Nix system, so provide one per system and update it when the dependency set, Lumis version, or artifact-selection settings such as LUMIS_USE_LEGACY_ARTIFACTS change. Use preConfigure, not preBuild: mixRelease compiles dependencies during its configure phase.

For a portable Linux x86_64 release, export LUMIS_USE_LEGACY_ARTIFACTS=true in both phases so the NIF does not require AVX/FMA support from the runtime host.

Building the NIF from source is also possible with config :rustler_precompiled, :force_build, lumis: true, but the Nix build must then provide Rustler, a Rust toolchain, and an offline Cargo dependency source. Prefetching the released NIF is usually simpler.

This NIF build cache is separate from the compiled-module cache described above, and from parsers entirely: the release contains the NIF, and parsers are dependencies inside it. Neither needs the network at runtime.

Next Page → CHANGELOG