Installing Firecracker on Ubuntu 26.04 LTS (development & testing)

This guide sets up Firecracker mode — the hardware-isolation boundary — on a Ubuntu 26.04 LTS host so you can boot and validate real neurons. It is written for a dedicated dev/test host (bare metal or a nested-virt-enabled VM), since Firecracker mode runs the orchestrator as root and needs /dev/kvm.

⚠️ Firecracker mode is implemented but not yet validated end-to-end (it can't run on macOS, where most development happens). This guide is how to bring it up and run the validation checklist. Until that checklist passes on your host, treat the boundary as unverified and run only trusted workloads. See Security Posture.

On macOS/Windows or any host without KVM, use dev mode instead (NSYS_CORTEX_MODE=dev + NSYS_CORTEX_ALLOW_INSECURE_DEV=1) — dev mode is not a security boundary; see Configuration → Security.

Supported hosts (architecture)

Firecracker's requirement is KVM on Linux, independent of CPU architecture:

Firecracker needs KVM on Linux, independent of CPU architecture:

arm64 itself is fully supported — the blocker is only macOS as the host OS and whether nested virt is available to a Linux guest. On an M1/M2 Mac (no nested virt), use Docker mode instead — a hardened container inside Docker Desktop's own Linux VM, no KVM required.

The decisive check — run inside the Linux guest; if /dev/kvm is present the guide works from step 2:

sudo apt-get install -y cpu-checker && kvm-ok && ls -l /dev/kvm

If /dev/kvm is missing: on an M3+/macOS 15+ Mac, enable nested virtualization in your hypervisor's VM settings (VM must be powered off to toggle) and re-check; on M1/M2 it can't be enabled — use dev mode on the Mac and run Firecracker validation on a separate Linux host (a cloud arm64 / Graviton instance matches an Apple-Silicon developer's architecture). Note nested KVM adds overhead, so boot is slower than bare-metal Linux — fine for testing, not representative of production timing.

What you'll install

The repo scripts pin and install everything:

• Firecracker v1.7.0 + jailer → /usr/local/bin/ (via scripts/setup-firecracker.sh)
• Guest kernel vmlinux-6.1.102 (Firecracker CI kernel) → ./resources/vmlinux
• Root filesystem built from rootfs/Dockerfile.rootfs → ./resources/rootfs.ext4

Ubuntu 26.04 is a good fit: it boots the unified cgroup v2 hierarchy by default (required for the jailer's cgroup caps) and ships iptables (nft backend) for the outbound NAT.

1. Verify KVM

sudo apt-get update
sudo apt-get install -y cpu-checker qemu-system-x86   # kvm-ok + kvm modules
kvm-ok                                                # expect "KVM acceleration can be used"
ls -l /dev/kvm                                        # must exist

If this host is itself a VM, enable nested virtualization on the hypervisor first (e.g. on a KVM host: options kvm_intel nested=1 / kvm_amd nested=1; on cloud providers, enable the nested-virt flag), or /dev/kvm won't appear.

2. Install host dependencies

# Networking (outbound NAT), container build for the rootfs, and git.
sudo apt-get install -y iptables docker.io git curl

# Node.js 24 (matches the repo's engines floor). Via NodeSource:
curl -fsSL https://deb.nodesource.com/setup_24.x | sudo -E bash -
sudo apt-get install -y nodejs
node --version    # v24.x

3. Get the code

git clone https://gitlab.nsysware.com/nsys/nsys-cortex.git
cd nsys-cortex
npm install
npm run build:agent          # required before building the rootfs

4. Install Firecracker + jailer + kernel, and provision the host

setup-firecracker.sh downloads the pinned Firecracker/jailer/kernel and provisions the jailer prerequisites (chroot base, cgroup delegation, IP forwarding) — the provisioning only runs when the script is root, so use sudo:

sudo -E bash scripts/setup-firecracker.sh

This installs firecracker/jailer to /usr/local/bin, downloads the kernel to ./resources/vmlinux, creates the jailer chroot base at $HOME/.nsys-cortex/jailer (0700 — /root/… under plain sudo, your own home under sudo -E; see the note below), delegates the cpu memory pids cgroup-v2 controllers both into the root and the nsys-cortex parent cgroup's subtree_control (the jailer's per-neuron leaf only gets memory.max/cpu.max/pids.max when the parent delegates them), and persists net.ipv4.ip_forward=1. Re-run any time; it's idempotent.

Chroot base co-location: the read-write overlay is hard-linked into the jail so guest writes persist across stop/start. That requires the chroot base (NSYS_CORTEX_CHROOT_BASE, default ~/.nsys-cortex/jailer) and the overlay store (NSYS_CORTEX_OVERLAY_STORE_DIR, default ~/.nsys-cortex/overlays) to be on the same filesystem — the defaults are. If you relocate one to a different mount, the neuron refuses to boot (EXDEV) rather than silently copy the overlay and lose the write-back on cleanup.

Same $HOME for setup and runtime: the orchestrator resolves ~ from $HOME (os.homedir()) for its config, overlay store and chroot base. sudo -E preserves your user's HOME; plain sudo switches it to /root. Run setup-firecracker.sh with the same form you'll start the orchestrator with (the commands in §6–7 use sudo -E, so sudo -E bash scripts/setup-firecracker.sh matches), or export NSYS_CORTEX_CHROOT_BASE explicitly for both. The script prints the base it provisioned.

Download integrity: the script prints the Firecracker tarball's SHA256 and, if you export FC_SHA256_X86_64 / FC_SHA256_AARCH64, verifies against it and aborts on mismatch; likewise FC_KERNEL_SHA256 for the guest kernel. Pin both digests from the release page before trusting them with root.

cgroup delegation in production: running the orchestrator directly as root (below) works with the manual delegation the script does. For a long-running service, run it under a systemd unit with Delegate=yes so systemd owns the cgroup subtree — otherwise systemd may reclaim the controllers. See Troubleshooting.

5. Build the guest rootfs

Builds rootfs/Dockerfile.rootfs (Alpine + Node 24 + socat/su-exec/util-linux

• the agent bundle) and exports it to an ext4 image. Needs Docker and root (loop mount + mkfs.ext4):

sudo bash scripts/build-rootfs.sh     # → ./resources/rootfs.ext4

6. Run in Firecracker mode

The install/build scripts write resources to the repo's ./resources, but the runtime defaults to ~/.nsys-cortex/resources. Point it at the repo copy (or move the two files). Firecracker mode must run as root (the runtime self-check enforces it):

export NSYS_CORTEX_RESOURCES_DIR="$PWD/resources"
export NSYS_CORTEX_MODE=firecracker

# HTTP API (add NSYS_CORTEX_API_TOKEN if binding beyond loopback):
sudo -E env "PATH=$PATH" npm run start

On startup the self-check fails closed — it refuses to start unless /dev/kvm, firecracker, jailer, root, cgroup v2 (cpu/memory/pids), and (when networking is enabled) iptables are all present, so you never silently run without the boundary. Tune isolation via the NSYS_CORTEX_* env vars in .env.example / Configuration (neuron ceiling, memory/pids caps, uid base, NSYS_CORTEX_ENABLE_NETWORK, etc.).

7. Smoke-test and validate the boundary

# Create a neuron and run a command in it (auto-creates on exec):
sudo -E env "PATH=$PATH" NSYS_CORTEX_MODE=firecracker NSYS_CORTEX_RESOURCES_DIR="$PWD/resources" \
  npx nsys-cortex exec test-neuron -- 'id; hostname; cat /proc/1/cmdline | tr "\0" " "'

Then walk the validation checklist (from the implementation plan / CHANGELOG):

1. Kernel features — inside a neuron: ls /dev/vsock exists, socat -V | grep -i vsock shows WITH_VSOCK. If missing, the guest kernel lacks virtio-vsock; rebuild vmlinux with CONFIG_VSOCKETS+CONFIG_VIRTIO_VSOCKETS.
2. Boot + persistence — the neuron reaches running; write a file under /home/agent/workspace, destroy+recreate the neuron (same id), confirm it persists.
3. Jailer isolation — on the host, cat /sys/fs/cgroup/nsys-cortex/<id>/memory.max shows your cap; a fork/stress bomb in the guest is contained; ls /root/.nsys-cortex/jailer/firecracker/<id>/root/run/ shows the API + cortex-vsock.sock_9999 sockets (nothing in /tmp).
4. Network isolation — the guest reaches the internet (curl https://example.com) but cannot reach the host, a sibling neuron, the LAN, or 169.254.169.254; no new inbound connects.
5. Gated integration test — sudo NSYS_CORTEX_TEST_FIRECRACKER=1 npm test. Once green, wire it into a KVM CI runner and update the security docs from "pending validation" to a validated boundary.

Troubleshooting

• Firecracker mode must run as root — the self-check rejects non-root; jailer needs root to build the chroot/cgroups and drop privileges. Use sudo -E.
• cgroup v2 not delegated with cpu/memory/pids — the parent cgroup lacks the controllers. Re-run sudo bash scripts/setup-firecracker.sh, or run under a systemd unit with Delegate=yes:

  # /etc/systemd/system/nsys-cortex.service
  [Service]
  Delegate=yes
  ExecStart=/usr/bin/npm run start
  Environment=NSYS_CORTEX_MODE=firecracker NSYS_CORTEX_RESOURCES_DIR=/opt/nsys-cortex/resources
  

• Guest has no IP / pivot_root fails — the CI kernel (vmlinux-6.1.102) may lack CONFIG_IP_PNP (needed for the ip= boot arg) or the config needed for pivot_root. Watch the neuron's serial console; if ip= doesn't configure the guest, use a kernel built with CONFIG_IP_PNP, or set NSYS_CORTEX_ENABLE_NETWORK=false to run neurons offline.
• iptables rules don't stick / teardown errors — Ubuntu uses the nft backend (iptables-nft); confirm iptables --version shows (nf_tables) and that add/delete round-trips. Legacy scripts on the host can conflict.
• Rootfs build fails on mount -o loop — run scripts/build-rootfs.sh as root; ensure loop module is available (sudo modprobe loop).
• /dev/kvm missing in a VM — nested virtualization isn't enabled on the hypervisor (see step 1).

Security caveat

Outbound internet is a data-exfiltration channel for a prompt-injected agent — the VM boundary protects the host, not data confidentiality. The egress firewall blocks lateral movement (host/LAN/siblings/metadata) but not exfil to the open internet. For sensitive workloads, front egress with a filtering proxy / DNS allow-list (a tracked follow-up). To run neurons with no network at all, set NSYS_CORTEX_ENABLE_NETWORK=false.