Blog

Hive: packages and languages, signed twice

Larva installs two kinds of code from the network. Packages are libraries your program compiles against. Language packs are compilers that run inside larva itself. Both are signed, and both are checked before anything is used. They are not trusted the same way, though, because they can do different amounts of harm. This post walks through both models, as Larva 0.8 implements them.

Packages in one minute

Hive is built into larva as larva hive. A package is a library named Author/Name, such as Antril-Organization/SDL3. It lives in a registry: the public one at https://www.antril.org/hive-api, another HTTPS server, or a plain folder. Adding one to a project is a single command:

Terminal
larva hive add Ana/Greetings@^1.0
added `package: Ana/Greetings@^1.0` to Hello.queen
  fetching Ana/Greetings 1.0 from C:\work\registry
  Ana/Greetings 1.0
installed 1 package(s); pinned in hive.lock

This adds a package: line to the project's DEPS chunk, installs the package into the user's Hive cache and writes hive.lock next to the .queen file. From then on, the package's namespaces are available:

Antril
import System;
import Greetings;

Main()
{
	Console.WriteLine(Greeter.Greet("Hive"));
}

Two rules shape everything else. Building never touches the network: larva build, larva run and the editor only look in the cache. And installing runs nothing from the package: there are no install scripts. Files are downloaded, checked and unpacked, and only later builds read them.

Making a package

A package is made from a library project. Here is the whole library behind Ana/Greetings:

Antril
into namespace Greetings
{
	/// Friendly greetings.
	public class Greeter
	{
		public:
			/// A greeting for `name`.
			static Greet(str name) : str => "Hello, " + name + "!";
	}
}

Its .queen file says what the package is called and how it is signed:

.queen
queen 1.0;

chunk PROJ:
   name: Greetings,
   type: library,
   version: v1.0,
   author: Ana,
   description: "Friendly greetings."

chunk HIVE:
   provider: ../registry,
   license: MIT,
   sign: hive.key

The key comes from larva hive key. It is an ECDSA P-256 key in a PEM file. Its fingerprint, the SHA-256 of the public key, identifies it everywhere:

Terminal
larva hive key
wrote C:\work\Greetings\hive.key: keep it private (sign packages with `sign: hive.key` in HIVE)
public key:  MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEt2nz5PDcKGyCEddkuAPfBauYEJVK4G3bfrFHnPyxn3r4UkfuUNsWCkhPMgBcQ3LBxGvu5O297PxsXhRrYvQ1UQ==
fingerprint: a59d969b683ba7f96e9fece9d1bcf78180354d077be31daeb5b9f2ebdc7ddb2d

Then larva hive pack builds the .hive file, and larva hive publish packs it and sends it to the project's first registry. Publishing to www.antril.org needs an account, an author you own (the Ana in Ana/Greetings) and an API token in the HIVE_TOKEN environment variable. Accounts, tokens and authors explains all three.

A package is either compiled, the default, carrying a Larva library built for one GC mode, or source, carrying the sources so that your project compiles them with its own settings. It can also wrap a library built elsewhere. The SDL3 package, for example, carries a generated binding and SDL3.dll, and a project that adds it gets SDL3.dll copied next to its program. See Compiled and source packages.

What a .hive file is

A .hive file is a zip archive with a fixed shape:

hive.json            the manifest
hive.sig             the signature of the manifest's exact bytes
lib/Greetings.dll    the files, exactly those the manifest lists
package.queen

hive.json lists every file with its size and SHA-256. hive.sig is an ECDSA P-256 signature over the manifest's bytes. Because the manifest holds the hash of every file, one signature vouches for the whole package. Hive signs the bytes as they are stored, not a re-serialized object, so a verifier never needs to reproduce anybody's JSON formatting. It hashes what is in the archive.

Packing is reproducible. Entries are sorted and every timestamp is fixed, so the same project always gives the same files. Only the signature differs, because ECDSA signatures are randomized. You can check any package file yourself:

Terminal
larva hive verify bin/Greetings-1.0.hive
Ana/Greetings 1.0 (compiled, gc default): 2 files, 4269 bytes
  signed by a59d969b683ba7f9 (not checked: give --key)
  every file matches its hash

With --key and the author's public key, verify checks the signature as well.

The registry's rules

A registry is mostly files: for each package an index.json and the package files beside it. The index lists each version with its SHA-256, the fingerprint of the key that signed it and its dependencies. It also lists the public keys allowed to sign the package. A folder registry and the www.antril.org registry apply the same rules when a version is published:

  • A version is published once and never replaced.
  • A package's first version brings its key.
  • Later versions must be signed by a key the package already has.
  • Once a package is signed, an unsigned version is refused.

So a registry cannot be used to switch a package's signer quietly. It also means your key matters. If you lose it, the registry will not accept new versions signed by another one. Signing and keys has the details.

What installing checks

When larva hive install fetches a version, it checks all of this before writing a single file to disk:

  • The file's SHA-256 is the one the registry's index gives. For larva hive restore, it must also be the one hive.lock pinned.
  • The file is a valid package. Its manifest lists every file, every file matches its size and hash, nothing unlisted is present, no path leaves the package's folder, and it stays within the limits (512 MB unpacked, 20,000 files). Sizes are enforced while decompressing, so an entry that claims to be small cannot expand without limit.
  • It is the package that was asked for: the manifest's name and version are the index's.
  • The signature verifies against the key the index gives for that version.
  • That key is one you trust for the package.

The last check is trust on first use. The first key a package is installed with is remembered in the cache, in keys.json. If a later version arrives signed by a different key, even from a different registry, installing stops and says what changed:

Terminal
larva hive install
  fetching Ana/Greetings 2.1 from C:\work\mirror
larva hive: Ana/Greetings is now signed by key bc0f3c915cb3c06d, not by the key it was installed with (a59d969b683ba7f9). If its author changed keys, accept the new one with `hive trust Ana/Greetings bc0f3c915cb3c06db2e910a4f5ae662e9a89db2664453d54b7e28012c7190ec0`

If the author really did change keys (check the fingerprint with them), larva hive trust accepts the new one. The old key stays trusted too.

By default packages must be signed. A project can accept unsigned packages with verify: hashes in its HIVE chunk. Hashes are still checked, and a version the index says is signed is still checked against its key.

hive.lock records, for each package, the exact version, the SHA-256 of its file, its signer's fingerprint and the registry it came from. Keep it with your sources. On a new machine or a build server, larva hive restore installs exactly what it pins and refuses any file whose hash is different.

Language packs: signed twice

A package is data that your compiler reads. A language pack is code that runs inside larva's own process. It is the frontend that turns source text into Larva's program model, plus the editor services behind larva lsp. Larva therefore asks more of a pack.

Even the packs that ship with Larva are checked every time. A pack's manifest, language.json, lists the SHA-256 of every file. Before loading a pack, larva checks that every listed file has its hash, and that no .dll, .so, .dylib or .exe in the pack's folder is missing from the list. Code the manifest does not name is never loaded. If the manifest is signed, the signature must match it. larva languages shows the result for each installed pack:

  status:    ok (files verified)
  status:    NOT USABLE: extra.dll is not among its `files`
  status:    NOT USABLE: its signature does not match its manifest (changed since it was signed)

Packs from the languages registry at https://www.antril.org/languages-api/ go one step further. Each published pack carries two signatures, and larva checks both before it loads a pack it installed:

  1. The author's signature. The author makes it with larva languages pack --key author.pem. It covers the canonical manifest without signature and without registry. The first time larva installs a language, it remembers the author's key fingerprint for that language id, in trust.json. After that, it accepts only versions signed by the same key.
  2. The registry's signature. antril.org adds it, as a registry object in the manifest, when an administrator approves the publication. It covers the canonical manifest without registry, so the author's signature is part of what it signs. larva ships with the registry's public key and refuses an installed pack without a valid registry signature.
json
{
  "id": "mylang",
  "files": { "MyLang.dll": "<sha-256>" },
  "publicKey": "<author's key, base64 SubjectPublicKeyInfo>",
  "fingerprint": "<sha-256 of the author's key>",
  "signature": "<author's ECDSA P-256 signature>",
  "registry": {
    "fingerprint": "<sha-256 of the registry's key>",
    "signature": "<registry's ECDSA P-256 signature>"
  }
}

(The real manifest also has name, version, abi, larva, entry and extensions. See The languages registry.)

The result is simple to state: a pack must be both approved by antril.org and made by the language's original author. A stolen author key is not enough without a review. A compromised registry is not enough without the author's key, at least for anyone who already has the language installed.

Unlike hive.json, a language manifest is signed in a canonical form. Keys are sorted by ordinal comparison, there is no whitespace, and strings are escaped the way a strict JSON writer escapes them for safe embedding. One consequence is that +, which appears in every base64 key and signature, is always written \u002B. Because files and publicKey are inside the signed text, changing a library, adding one or swapping the key all break a hash or the signature.

Publishing a language

Anyone can ask to publish a language, and an administrator of antril.org reviews each request by hand. In short:

  1. Make a key once with larva languages key author.pem, and keep it private. Every version must be signed with it.
  2. Build the pack for each platform you support, one folder per platform (win-x64, linux-x64, osx-arm64…).
  3. Send the request with larva languages publish packs/ --key author.pem and your API token in HIVE_TOKEN.

The first request for a language id makes you its owner and pins your key to it. Some ids are reserved, such as antril. larva languages status <request> and your account page show where a request stands. Users then install with larva languages install mylang, and update and remove work as you would expect. remove keeps the remembered author key unless you add --forget-key, for when an author really changed keys.

The languages registry is new in this release and still changing. Teams can run a private one, a folder with the same layout signed by their own key, using larva languages approve and larva config languages.trust. See Registries in the internals book.

Not there yet

  • Yanking: the index has a yanked field and installers honor it, but nothing sets it on www.antril.org yet.
  • An author has owners only. Adding other people to an author is not available yet.

Read more

Browse the packages on Hive, and bring questions to the forum.

All posts · Atom feed · RSS

Esc