Native executables and libraries

A back end (LLVM or x64) makes an object file. Turning that object into something the operating system runs takes three more things: a function the C side can call to start the program, the native runtime (larva_rt.c), and the system's linker. That is the job of Larva.Driver, with the toolchain classes of Larva.Native.

The path of larva build

 larva build -o app.exe app.ant
   frontend -> HIR link -> lowering -> IR checks -> IR link             (see Projects and builds)
        |
        v
 ExecutableBuilder.Build(linked, "app.exe", options)
   1. Toolchain.Find()                     MSVC, or "no C toolchain: ..."
   2. ExecutableEntry.AddWrappers          larva_entry, larva_entry_f64, larva_entry_info
   3. back end -> object(s)                %TEMP%\larva-exe-<guid>\program.obj  (or program.<p>.obj ...)
   4. Toolchain.Link                       objects + cached larva_rt-<hash>.obj + kernel32, ws2_32, ntdll
   5. delete the temporary folder
        |
        v
 app.exe   "app.exe (llvm back end, O3, entry app::...Main)"

Every native build is ahead of time: the object files are written to a temporary folder, linked into the output, and the folder is removed. Nothing runs inside larva's own process (the in-memory path of the LLVM back end exists as a library API; no larva command uses it).

The native (llvm) or native (x64) phase of --timings covers steps 1 to 5. With LARVA_TIMINGS=1 its parts show up:

$ LARVA_TIMINGS=1 larva build --timings --backend x64 -o hello64.exe hello.ant
[timing] type check: 86 ms
[timing] region check: 0 ms
[timing] x64: generate: 19 ms
[timing] link (runtime and linker): 383 ms
hello64.exe (x64 back end, entry hello64::Hello.$Module.Main)
timings: System.Core (compiled library) 8 ms, frontend 83 ms, HIR link 8 ms, lowering 117 ms, IR checks 87 ms, IR link 45 ms, native (x64) 511 ms; total 859 ms
$ LARVA_TIMINGS=1 larva build --timings -o words.exe words.ant
[timing] type check: 87 ms
[timing] region check: 0 ms
[timing] monomorphize (specialize): 6 ms
[timing] llvm: build IR (89 functions): 23 ms
[timing] llvm: verify: 3 ms
[timing] llvm: optimize (O3): 480 ms
[timing] llvm: emit object: 433 ms
[timing] link (runtime and linker): 380 ms
words.exe (llvm back end, O3, entry words::Words.$Module.Main)
timings: System.Core (compiled library) 6 ms, frontend 111 ms, HIR link 17 ms, lowering 171 ms, IR checks 88 ms, IR link 47 ms, native (llvm) 1457 ms; total 1897 ms

The link step is the same for both back ends, a few hundred milliseconds. The x64 back end generates code in tens of milliseconds where LLVM spends most of a second optimizing and emitting. The resulting Hello World is 68,096 bytes from LLVM at -O3, 85,504 bytes at -O0, and 109,056 bytes from the x64 back end.

The entry point

The C runtime owns the process entry. It calls the program through three functions that ExecutableEntry.AddWrappers adds to the linked IR (they become link roots, and keep their unprefixed larva_ names):

The bits of larva_entry_info:

The runtime's program_main reads its settings from the environment, calls larva_entry_info, and checks that the command line has exactly arity words, each parsed as an integer (true/True and false/False are 1 and 0). It calls the entry under a handler that reports an uncaught Larva exception, then prints the result according to its kind: %lld, %llu, True/False, or %.17g.

What Main can look like

AddWrappers accepts an entry that is not generic, takes at most four parameters, each an integer, a bool or a float, and returns nothing, an integer, a bool or a float. Anything else stops the build.

import System;

into namespace Square
{
	Main(i64 n) : i64
	{
		return n * n;
	}
}

$ larva build -o num.exe num.ant
num.exe (llvm back end, O3, entry num::Square.$Module.Main)
$ ./num.exe 12
144

Without its argument, num.exe prints its usage line and exits with code 2.

Known gaps

Main(str[] args) cannot be an entry. The type check accepts it, but the executable builder does not:

import System;

into namespace Args
{
	Main(str[] args)
	{
		Console.WriteLine(args.Length);
	}
}

$ larva build -o args.exe args.ant
larva: entry `args::Args.$Module.Main` takes a `gc [gc struct#128<u8>]`; executables pass integers, bools or floats

System.Environment.GetCommandLineArgs() exists, but it cannot be used either: a program reads its own command line only when its entry carries the Larva.Program attribute (bit 16 above), and no frontend sets it today. A Main() without parameters therefore rejects any argument: ./app.exe one two prints usage: … (0 integer arguments) and exits with code 2.

Main() : i32 prints its result instead of exiting with it. Kind 5 (exit code) is only chosen together with the Larva.Program attribute; an ordinary integer result is kind 1, printed, and the process exits with 0:

import System;

into namespace Codes
{
	Main() : i32
	{
		Console.WriteLine("done");
		return 3;
	}
}

$ larva build -o code.exe code.ant
code.exe (llvm back end, O3, entry code::Codes.$Module.Main)
$ ./code.exe
done
3
$ echo $?
0

The WebAssembly runtime follows exactly the same rules.

Symbols

Larva functions are prefixed so they cannot clash with C symbols such as main (NativeSymbols.Of, ExecutableBuilder.SymbolPrefix = "larva."):

The hash keeps symbols stable between builds (libraries keep linking), short, and free of characters an export list cannot hold.

The C runtime: larva_rt.c

Larva.Native/Runtime/larva_rt.c (about 7,300 lines) is embedded in larva and compiled for each machine the first time it is needed. It holds the garbage collectors, heap cells, arrays, regions, exceptions, threads, files, sockets, processes, printing and parsing of numbers, and the soft-float and wide-integer routines. See The runtime and Garbage collection.

On Windows it links no C library. Programs take only kernel32, ws2_32 and ntdll from the system:

• memory comes from the process heap (HeapAlloc);
• standard input, output, error and files are handles with a buffer of their own (WriteFile, ReadFile), in text mode as the C library had them;
• printf and friends, and conversions between doubles and text (exact, correctly rounded), are written in the runtime;
• memcpy, memmove and memset are the runtime's own (faster than ntdll's); memcmp, strlen, wcslen, strrchr, __chkstk and __C_specific_handler come from ntdll.dll;
• the transcendental math functions (sin, exp, pow, …) are loaded from the system's ucrtbase.dll the first time one is used, so results match other programs on the machine; sqrt, floor and ceil are in the runtime;
• the entry point, larva_crt_start, splits the command line exactly as the C library would (quotes, backslashes, "" inside quotes), in UTF-8, and calls main.

The design notes give the reason: the static C library was most of every program (printf, strtod, locale and logarithm tables). With it gone and nothing taken from a redistributable, an executable runs on any 64-bit Windows 10 or later as a single file. LARVA_WITH_CRT=1 builds the old way, against the Windows UCRT, to compare.

Compiled once, cached

Toolchain.CachedRuntime compiles the runtime with

cl /nologo /O2 /Gy /Gw /GS- /W3 /c [/DLARVA_WITH_CRT] [/DLARVA_DIAGNOSTICS] /Fo<partial> larva_rt.c

into %LOCALAPPDATA%\Larva\toolchain\larva_rt-<hash>.obj, where the hash covers the runtime source, the flags and the path of the compiler's setup script. A new version of larva, LARVA_DIAGNOSTICS=1 or a different Visual Studio therefore gets its own object; every later build only links. The object is written under a temporary name and then moved into place, so two builds running at once never see half a file (if another build got there first, its identical file is used).

The same folder caches two more things:

$ ls "$LOCALAPPDATA/Larva/toolchain"
larva_rt-2E1A08C57DB4E57D5BFD.obj
larva_rt-F301DD493A3C575D3D96.obj
ntdll-crt-1344D537C86D61F549B9.lib
ntdll-crt-1344D537C86D61F549B9.lib.2f3bb644e5c0443f8404d69948dbae81.exp
vcvars-F26404706C724FBE5714.env

(Two runtime objects: two larva versions have built on this machine.) LARVA_DIAGNOSTICS=1 matters for anyone measuring the collector: only a runtime compiled with it reads LARVA_GC_STRESS, LARVA_GC_VERIFY, LARVA_STATS and the other diagnostic variables at run time; without it, that code is left out and programs are smaller.

Finding the linker on Windows

Toolchain.Find looks for MSVC (cl, link, lib) and nothing else:

1. If LARVA_VCVARS is set, it must be the path of a vcvars64.bat. If that file does not exist, there is no toolchain (nothing else is tried).
2. Otherwise it runs %ProgramFiles(x86)%\Microsoft Visual Studio\Installer\vswhere.exe -latest -prerelease -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath: any Visual Studio or Build Tools, stable or preview, with the C++ tools.
3. The toolchain is <installation>\VC\Auxiliary\Build\vcvars64.bat.

$ LARVA_VCVARS='C:\nowhere\vcvars64.bat' larva build -o nope.exe hello.ant
larva: no C toolchain: install Visual Studio Build Tools with the "Desktop development with C++" workload (MSVC and the Windows SDK), or set LARVA_VCVARS to the path of vcvars64.bat

With the cached environment, link.exe is found on its PATH and started directly, with no batch script and no cmd.exe in between. Only when the environment cannot be read does larva fall back to a script that calls vcvars64.bat first. The runtime source contains code paths for clang-cl too, but larva itself only drives MSVC on Windows.

The link line

For an executable:

link /nologo /CETCOMPAT:NO /SUBSYSTEM:CONSOLE /ENTRY:larva_crt_start /OPT:REF /OPT:ICF /INCREMENTAL:NO /NODEFAULTLIB
     /OUT:app.exe program.obj larva_rt-<hash>.obj kernel32.lib ws2_32.lib ntdll-crt-<hash>.lib

Libraries for C: --dll

larva build --dll builds a DLL that C (or any language with a C foreign-function interface) can load. Its functions are the program's [Export] functions, under their C names:

import System;

into namespace MathLib
{
	[Export("add_numbers")]
	public static AddNumbers(i32 a, i32 b) : i32 => a + b;

	[Export]
	public static Square(f64 x) : f64 => x * x;
}

$ larva build --timings --dll -o mathlib.dll mathlib.ant
mathlib.dll (llvm back end, O3, exports add_numbers, Square)
timings: System.Core (compiled library) 8 ms, frontend 87 ms, HIR link 11 ms, lowering 137 ms, IR checks 80 ms, IR link 46 ms, native (llvm) 1117 ms; total 1486 ms
$ larva bind --list mathlib.dll
Square
add_numbers

[Export] without a name uses the function's own name. The linker also writes mathlib.lib (the import library a C program links against) and mathlib.exp next to the DLL. The x64 back end builds DLLs too (--dll --backend x64).

The Antril frontend checks the exported functions: an export is a static function with a body and no type parameters (error AN1055), and takes no str (AN1056).

What differs from an executable:

• The program is linked with LinkOptions.CLibrary: the exported functions are the roots, and a Main is optional.
• ExecutableEntry.AddLibraryStubs adds larva_entry, larva_entry_f64 and larva_entry_info returning 0, because the runtime object still names them.
• The link line is link /nologo /DLL /ENTRY:DllMain /CETCOMPAT:NO /OPT:REF /OPT:ICF /INCREMENTAL:NO /NODEFAULTLIB /OUT:… <objects> <runtime> kernel32.lib ws2_32.lib ntdll-crt.lib /EXPORT:<name>…. The runtime's DllMain reads the runtime settings when the DLL is loaded.
• The runtime is inside the DLL. Two Larva DLLs loaded into one process each carry their own runtime and heap.

Programs split into DLLs

NativeLibraryBuilder builds a different kind of program: one DLL per Larva namespace (NAME.dll), all sharing one runtime DLL, larva_rt.dll, and a program that loads them.

 app.exe  ----imports---->  Geometry.dll  ----imports---->  larva_rt.dll  (the runtime, with its C library)
    |                            |
    +-------imports--------------+------> System.dll ...

• BuildLibraries links the objects together as a library, then for each of its own modules: NativeImages.Prepare keeps that module's code and declares what the others define; the monomorphizer assigns id slots (type ids and the like must agree across images), NativeImages.AddIdSetter adds the function that fills them; the back end compiles it; link /DLL /NOENTRY exports its symbols against larva_rt.lib and the import libraries of the others. Import libraries are made from export lists alone (lib /def), so DLLs can depend on each other in any order. It returns the objects with their id manifests, to be saved as the .lvo files that go with the DLLs.
• BuildProgram links a program against those DLLs: a small C start file (larva_start) hands the entry wrappers to larva_run_main in larva_rt.dll, which parses the command line (with CommandLineToArgvW) and runs program_main. The DLLs and larva_rt.dll are copied next to the program.
• larva_rt.dll is compiled with /MT /DLARVA_SHARED_RUNTIME and keeps the static C library, so neither the program nor the Larva DLLs need one.

Not implemented yet: no larva command builds programs this way. larva build always produces one self-contained executable or one C-facing DLL; NativeLibraryBuilder is a library API of Larva.Driver.

Linux and macOS

PosixBuilder and PosixToolchain are the counterparts for Unix-like systems. Code always goes through LLVM (the direct x86-64 back end speaks only the Windows ABI), with the LLVM triple of a NativeTarget:

The C compiler is cc (gcc or clang). PosixToolchain.Find runs cc --version on Linux or macOS. On Windows, for Linux targets, it runs wsl.exe -e cc --version instead and, if that works, compiles and links inside WSL, translating C:\x to /mnt/c/x. Its messages say what is missing: no C compiler: install gcc or clang (cc), WSL has no C compiler: run sudo apt install build-essential in it, or that macOS programs are built on a Mac.

The runtime is not cached there: larva_rt.c is compiled into each executable in the same cc call.

The rpath makes each library look for the others next to itself. Code for these targets is position-independent, uses the exception flag instead of SEH, and has no stack maps: the runtime spills the callee-saved registers and scans the thread's stack conservatively.

Not implemented yet: larva build does not reach PosixBuilder. Its native path always goes through the Windows Toolchain, so on Linux and macOS a native build stops with the "no C toolchain" message; WebAssembly builds work there. The POSIX runtime and builders are written but have not been run on a real Linux or macOS machine yet.

Environment variables

At run time, an executable built with LARVA_DIAGNOSTICS=1 reads LARVA_GC_STRESS, LARVA_GC_THRESHOLD, LARVA_GC_VERIFY, LARVA_GC_QUARANTINE, LARVA_GC_NOSMALL, LARVA_GC_GENERATIONS, LARVA_GC_NURSERY, LARVA_GC_THREADS and LARVA_STATS; one built without it reads none of them. See The runtime.