[DRAFT - held on OSv fork/COW landing] postgres18-musl: stock PostgreSQL 18 on OSv demo - #73
[DRAFT - held on OSv fork/COW landing] postgres18-musl: stock PostgreSQL 18 on OSv demo#73gburd wants to merge 6 commits into
Conversation
Builds PostgreSQL 18 (REL_18_STABLE) from source with musl-gcc as an OSv PIE, boots a small baked cluster, and serves SQL over TCP. Single-backend demo; requires the OSv fork/COW coherence work (conf_fork=1 + the fork-arena kernel fixes) which is not yet on OSv master, so this stays a WIP branch until that lands.
…ady-serving) Add a documented full-VM snapshot/restore capability to the demo: boot OSv+PostgreSQL under Firecracker to "ready to accept connections", take a Full snapshot (guest memory + device/vCPU state), and on later launches restore that snapshot so PostgreSQL is already serving. No reboot, no re-initdb, no musl-PIE re-mmap. Measured on m5d.metal (Firecracker v1.7.0, OSv v0.57 PG18.4-musl + OpenZFS, single-file ZFS pool recordsize=8k): cold launch -> first external select 1 = 11.75 s restore -> first external select 1 = 0.24 s (median of 5, ~48x) FC resume itself = 0.05 s The ~6.7 s PostgreSQL musl-PIE mmap+init is eliminated (baked into the snapshot). OSv resumes cleanly with NO OSv kernel changes required (already snapshot-ready). Proof it is a resume not a reboot: pg_postmaster_start_time() after restore is the original pre-snapshot boot time. Disk consistency via a fresh COW copy (cp --reflink=auto) of the pool at snapshot time so the guest ARC/shared_buffers agree with disk. Honest gotchas documented: clock skew (kvmclock re-syncs, ~50-85 ms stable offset, tolerated), virtio re-attach (host tap+DNAT must pre-exist), RNG (no observed duplication, weak probe). Files: fc-snapshot.sh (driver), fc-snap.py (Firecracker API helper), README section with build/snapshot/restore steps, measured numbers, and gotchas. Author: Greg Burd
Add a 'lifecycle' subcommand that proves a PERSISTENT WARM checkpoint: a first warm restore (~0.2s to first query), insert rows in the restored session, then FREEZE (a fresh Full snapshot plus a preserved ZFS backing file) followed by a CLEAN Firecracker shutdown (the VM process exits entirely), and finally a SECOND warm restore from the frozen state (~0.2s again). The table, its rows, and the original pg_postmaster_start_time() all survive the freeze/clean-shutdown/warm-restart cycle, so the instance identity and its data are a durable resumable checkpoint. Also add a scripted 'seed' subcommand (build the single-file ZFS pool and cpiod-push an initdb'd cluster into it) with its cpio_push.py helper, and fix the boot argument passing so the leading --rootfs is not mis-parsed by the Firecracker driver argument parser. Measured on m5d.metal with Firecracker v1.7 (4 GiB guest, single-file ZFS pool): both warm launches were about 0.17s to first query and pg_postmaster_start_time() stayed equal to the very first launch across every freeze/thaw cycle. Signed-off-by: Greg Burd <greg@burd.me>
GCC 13 libgcc lse-init.o references glibc __getauxval (absent in musl); every link fails and PG configure cannot detect the GCC atomic builtins. Disable the LSE runtime-detection path on aarch64 so links succeed and atomics are found.
… note Two unrelated one-line problems in the same module. The Makefile declared '.PHONY: module' but not 'build', while 'module: build' depends on a target named 'build' and the module also creates a real 'build/' directory. Make then considers 'build' up to date because the directory exists, skips the recipe, and 'module' succeeds without building anything. The result is an image that contains no PostgreSQL while the build reports success, which has silently produced a 10 MB image where a working one is about 65 MB. Declaring 'build' phony as well makes the recipe always run. Separately, the comment claimed OSv's worker AIO path is incomplete and lumped it together with io_uring. io_method=worker is verified working on OSv: PostgreSQL 18.6 starts, forks its three IO worker processes and serves queries, with SHOW io_method reporting 'worker' to an external client. It needs no kernel AIO, being shared memory plus a latch plus forked workers calling preadv/pwritev. io_uring is the incomplete one, and it also needs --with-liburing at configure time. Since io_method is PGC_POSTMASTER, one build can select any method at run time with -c io_method=..., so the demo default is left at sync and the comment now says why. Signed-off-by: Greg Burd <greg@burd.me>
|
Two one-line fixes pushed, both found while verifying PostgreSQL 18's asynchronous I/O on OSv. The Worth knowing as a check regardless of this fix: image size is a reliable smoke test here. A PostgreSQL-bearing The PostgreSQL started cleanly, forked its three IO worker processes, reported "database system is ready to accept connections", and served queries with no faults, with PGDATA on a ZFS dataset rather than a memory filesystem.
Since |
seed_copy faithfully preserved the source mode, which is correct on a filesystem that has modes. rofs does not: scripts/gen-rofs-img.py stores only the S_IFMT type bits (DIR_MODE 0x4000, REG_MODE 0x8000), discarding the permission field entirely, and fs/rofs/rofs_vnops.cc's rofs_getattr() then reports a hardcoded 0555 for every inode. So copying the observed mode created the destination tree without an owner-write bit, and the next level down failed: O_CREAT hit vn_access(VWRITE) and returned EACCES. Seeding a PostgreSQL cluster from a baked read-only image therefore stopped at the second directory level. A 0555 PGDATA would have been rejected by PostgreSQL regardless. Use the modes initdb itself produces, 0600 for files and 0700 for directories, rather than propagating modes the source filesystem cannot represent. With this, seeding a 19 GB 200-warehouse cluster onto a ZFS dataset completes with no errors. Signed-off-by: Greg Burd <greg@burd.me>
|
One more fix, found while seeding a PostgreSQL cluster onto ZFS inside an OSv guest.
DIR_MODE = int('0x4000', 16)
REG_MODE = int('0x8000', 16)
LINK_MODE = int('0xA000', 16)and assigns those unconditionally, so only the The consequence is that copying the observed mode created the destination tree without an owner-write bit, and the next level down failed: This uses the modes Worth noting separately, since it is a defect in the image tooling rather than in this module: rofs discarding permissions is invisible until something tries to write into a tree copied out of it. Anything that reads a mode from a rofs image is reading an invented value. |
Draft — held on the OSv fork/COW work landing on OSv master
Adds a
postgres18-musl/demo app that runs stock, unmodified PostgreSQL 18(musl build) on OSv.
This will NOT run on stock OSv master. It requires the OSv-side
thread-backed
fork()+ per-child COW address space + per-process signals + thefork-completeness stack (AS-aware mmap, fd inheritance, timer parking, cross-AS
coherence) to be present on OSv
masterfirst. Until then this is filed as adraft so the app is visible alongside the kernel work it depends on.
Requires (on OSv, not this repo)
fork()/vfork()/execve()/waitpid()fork()(S1..S6: AS-aware mmap, COW arena, fd inheritance, per-process signals,
timer parking, cross-AS coherence), and — for ZFS-backed
/data—zfs: selectable in-kernel ZFS (BSD or OpenZFS 2.4.2) via upstream submodule + patch series osv#1423 (selectable in-kernel OpenZFS).
All of the above are opt-in /
CONF_forkon OSv; PG-per-backend fork is thewhole reason this app needs them.
Status
Ready to draft; held on fork-merge. Do not merge until the OSv fork/COW
stack (#1455/#1456/#1457 + #1458) is on OSv
master; add ZFS (#1423) for theZFS-on-NVMe
/datavariant.