Loads Tree-sitter parsers this project depends on.
A language is a parser WASM plus the queries that drive highlighting, shipped
together as a lumis_wasm_* dependency. Nothing here needs calling for normal
use: Lumis.highlight/2 loads what a document turns out to need, including
languages injected inside it — as long as the project depends on them. One it
does not answers :not_installed.
Reach for it to load ahead of the first request, so the compile does not land
on a user. From an application's start/2, use async_load/1, which returns
without holding up the boot:
Lumis.Languages.async_load(["elixir", "html"])
:ok = Lumis.Languages.load(["elixir", "html"])Where parsers come from
From the priv/parsers of each installed lumis_wasm_* dependency, or a
directory named by config :lumis, :parser_dirs. A language no dependency
supplies is not fetched — it answers :not_installed. The bytes are loaded as
Hex delivered them, or as the project put them there.
Concurrent requests for the same unloaded language wait for the first rather than each compiling it, and loading is global to the VM: the process that pays for a language pays once, for every process after it.
Summary
Types
@type bundle() ::
:bundle_web
| :bundle_web_extra
| :bundle_system
| :bundle_backend
| :bundle_full
A language set shared with the @lumis-sh/wasm-bundle-* packages.
@type failure() :: :unknown_language | :not_installed | :failed_to_load_parser | Lumis.ParserError.t()
Functions
@spec async_load(bundle() | String.t() | atom() | [String.t() | atom()]) :: DynamicSupervisor.on_start_child()
Runs load/1 in the background and returns immediately.
Written for start/2, where the alternative is holding the whole boot on a
compile. Highlighting loads on demand anyway, so an application that starts
before its parsers are warm serves correctly the entire time; it only pays for
a language on the first request that names one the warm-up has not reached.
def start(_type, _args) do
Lumis.Languages.async_load(~w(markdown elixir javascript))
Supervisor.start_link(children(), strategy: :one_for_one, name: MyApp.Supervisor)
endFailures are logged rather than returned, because nothing is waiting on them.
Nothing this function does can stop an application from booting: the work runs
under a :temporary child of Lumis's own supervisor, so it is never retried
and never escalates. Use load/1 when the caller does need the result.
Returns {:ok, pid}; the docs above ignore it deliberately, since matching on
it is how a warm-up ends up able to break a boot after all.
The languages each :bundle_* name covers.
These are the same sets the @lumis-sh/wasm-bundle-* packages ship, so naming
a bundle means the same thing in every runtime.
@spec get(String.t() | atom(), any()) :: Lumis.language_info() | any()
Looks one language up by id or alias, or returns default.
The same record Lumis.available_languages/0 returns one of, without scanning
the catalog, and resolving aliases the way highlighting does: "js" finds
JavaScript.
iex> Lumis.Languages.get("js").id
"javascript"
iex> Lumis.Languages.get("not-a-language")
nil
Resolves a name, path or source to a language id, the way highlighting does.
name can be a language id, an alias, a file name or a path. When it does not
resolve, source is checked for an Emacs mode header, a shebang, an HTML
doctype or an XML declaration. Falls back to "plaintext".
The counterpart of Language::guess in Rust and guessLanguage() in
JavaScript. Lumis.highlight/2 already calls this when no language is given;
reach for it directly to label a snippet before deciding what to do with it.
"elixir" = Lumis.Languages.guess("lib/app.ex")
"bash" = Lumis.Languages.guess(nil, "#!/usr/bin/env bash")
"plaintext" = Lumis.Languages.guess(nil, "")
@spec load(bundle() | String.t() | atom() | [String.t() | atom()]) :: :ok | {:error, :unknown_bundle | failure()} | {:error, %{required(String.t()) => :unknown_bundle | failure()}}
Loads one language, a list of them, or a bundle.
Highlighting loads on demand, so this is an optimization rather than a requirement: call it at startup to move the compile off the first request. Already-loaded languages return immediately.
:ok = Lumis.Languages.load("elixir")
:ok = Lumis.Languages.load(["elixir", :html])
:ok = Lumis.Languages.load(:bundle_web)A :bundle_* atom names the same set of languages as the lumis_wasm_bundle_*
package of that name, so every runtime means the same thing by it. :bundle_full
is every language in the catalog, which is rarely what a deployment wants; prefer
a narrower bundle, or name the languages a document can contain.
Loading only ever reaches parsers this project depends on, because a parser is an ordinary dependency:
# mix.exs
{:lumis_wasm_elixir, "~> 0.26.0"}Failures
A list loads every name in it and reports the ones that failed, rather than stopping at the first. One unpublished parser in a bundle should not cost the rest, the same way one bad block does not cost a document.
{:error, %{"css" => :failed_to_load_parser}} = Lumis.Languages.load(["elixir", "css"]):unknown_language— the name is not in the catalog:not_installed— it is, but this project does not depend on its parser:failed_to_load_parser— the parser could not be read or loaded:unknown_bundle— no bundle by that name%Lumis.ParserError{reason: :store_full}— the parser is fine, and this process has no room for another one
The last is the exception Lumis.highlight/2 answers with, rather than an
atom, because it is the one failure here that is not about the language named:
it loads on its own, retrying it cannot help, and Exception.message/1 says
what does. See Lumis.ParserError for the reason itself.
A single name answers with the reason itself; a list answers with a map from name to reason, so one failure never hides the others.