Skip to content

binfmt/fdpic: Add an FDPIC ELF loader for execute-in-place modules - #19584

Closed
casaroli wants to merge 9 commits into
apache:masterfrom
casaroli:xipfs-fdpic
Closed

binfmt/fdpic: Add an FDPIC ELF loader for execute-in-place modules#19584
casaroli wants to merge 9 commits into
apache:masterfrom
casaroli:xipfs-fdpic

Conversation

@casaroli

@casaroli casaroli commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an FDPIC ELF loader so a downloadable module can execute in place out of memory-mapped flash: the read-only segment is mapped where it already sits on the media and never copied to RAM, and only the writable segment is copied, once per running instance.

The in-tree ELF loader already does shared .text with private .data on a no-MMU target (CONFIG_PIC, mps3-an547:picostest), so that part is not new. What FDPIC adds is a function pointer that carries its own data base, as a two-word descriptor {entry, GOT} rather than a bare code address. Two things follow that a single base register per task cannot express: a module can be called back on a thread it never created, such as the work-queue worker that runs a SIGEV_THREAD notification; and two objects can hold distinct data bases at once, which is what makes DT_NEEDED shared libraries work with per-instance library data.

Compared with NXFLAT: FDPIC is standard ELF, needs no mknxflat/ldnxflat and no linker script, puts .rodata in the RX segment on its own, supports shared libraries, and needs no workaround for a pointer to a static function. The cost is an arm-uclinuxfdpiceabi linker; the stock arm-none-eabi compiler emits correct FDPIC objects for both C and C++, so only the link needs it.

The loader is binfmt/fdpic.c under CONFIG_FDPIC (depends on CONFIG_ARCH_ARM, selects BINFMT_LOADABLE and PIC). It requires a filesystem that answers BIOC_XIPBASE, such as XIPFS or ROMFS. arch/arm/src/common/Toolchain.defs adds --fixed-r9 so the base firmware reserves the FDPIC register; without it a firmware callback into module code arrives with the wrong data base.

Nine commits: the DT_*_ARRAY tags in include/elf.h; the loader; constructors and PLT relocation binding; descriptor resolution at the libc/sched entry points that accept a module callback; SIGEV_THREAD notifications; the developer guide in Documentation/components/fdpic.rst; the out-of-tree module build tooling in tools/fdpic; and a board configuration plus the demo's documentation.

Paired with apache/nuttx-apps#3682, which adds the demo and the test suite sections. This PR stands alone and is the one to merge first: nothing in the apps PR can be enabled until CONFIG_FDPIC exists here. Merging this first leaves pimoroni-pico-2-plus:xipfs-fdpic building the loader and the filesystem tests but not the demo, because CONFIG_EXAMPLES_FDPICXIP does not resolve yet; that corrects itself when the apps PR lands, with no follow-up change needed.

Impact

New feature, off by default. CONFIG_FDPIC defaults to n, so a tree that does not enable it is unaffected.

Existing code touched in two places, both no-ops without CONFIG_FDPIC. First, ten libc/sched entry points that can accept a callback from a module now resolve an FDPIC descriptor before storing or branching to it: qsort, bsearch, pthread_create, signal, sigaction, task_create/task_create_with_stack, task_spawn, pthread_once, scandir, and mq_notify/timer_create with SIGEV_THREAD. Each is guarded by #ifdef CONFIG_FDPIC. Second, Toolchain.defs adds --fixed-r9 to ARCHCPUFLAGS, again only under CONFIG_FDPIC, which costs the base firmware one register in that configuration.

Adds one board configuration, pimoroni-pico-2-plus:xipfs-fdpic. It builds with a plain arm-none-eabi toolchain because the test modules are committed as prebuilt blobs, so CI covers it; it deliberately does not enable CONFIG_NXFLAT, which would need mknxflat and require excluding the configuration from the build list.

Documentation: one new page, Documentation/components/fdpic.rst, added to the components toctree, plus the fdpic and reject sections in the xipfs test suite page.

No change to any existing API, ABI, or default configuration. FDPIC is ARM Thumb-2 only; RISC-V has no FDPIC ABI, so a RISC-V target cannot use this loader.

Testing

Host: macOS 26.5.1 on arm64. Arm GNU Toolchain 15.2.Rel1 (arm-none-eabi-gcc 15.2.1), arm-uclinuxfdpiceabi-ld from GNU Binutils 2.43, QEMU 11.0.3.

QEMU, mps2-an500 (Cortex-M7, ARMv7E-M) — the loader on a different core generation from the hardware below, with xipfs on a rammtd device that answers BIOC_XIPBASE. Built from mps2-an500:xipfs plus the FDPIC options. Full xipfs_test suite:

==== 130 passed, 0 failed ====

The two FDPIC sections on their own:

nsh> xipfs_test fdpic
==== 33 passed, 0 failed ====

nsh> xipfs_test reject
-- FDPIC loader rejections --
  PASS  a non-32-bit ELF class is refused
  PASS  a non-ET_DYN object is refused
  PASS  a module for the wrong machine is refused
  PASS  a module with RELA PLT relocations is refused
  PASS  a module whose needed library is absent is refused
  PASS  a module importing an unexported symbol is refused
  PASS  a module with too many DT_NEEDED entries is refused
==== 7 passed, 0 failed ====

The fdpicxip demo, all four subcommands. solib, showing two instances sharing one copy of a library's text in flash while each gets its own copy of its data:

nsh> fdpicxip solib
=== FDPIC shared library, executed in place ===

staged libcounter.so (2052 bytes) and user (2732 bytes)
  libcounter.so text at 0x6000cd20
  user          text at 0x6000dd20

spawning two instances, each bumping by its own seed...

[while running] pins on the library's shared text = 2

[user 1] library total = 3 (expected 3) -- PASS
[user 2] library total = 6 (expected 6) -- PASS

[after exit]    pins on the library's shared text = 0

cxx additionally confirms each object's global constructors ran in dependency order before main, and jmprel that a module whose imports are all in DT_JMPREL binds and calls out.

Hardware, Pimoroni Pico Plus 2 (RP2350, Cortex-M33, ARMv8-M) against real QSPI flash, pimoroni-pico-2-plus:xipfs-fdpic: full suite 130/130, fdpic 33/33, reject 7/7, and all four demo subcommands. The same module blobs run on both cores; they are built for cortex-m3, so one set serves v7-M and v8-M.

Simulator, sim:xipfs on the host, which exercises the filesystem half only since the loader is ARM-only:

==== 90 passed, 0 failed ====

Loader assertions verified against deliberate breakage. Each check in the fdpic section was confirmed to fail when the thing it names is broken — removing the scandir filter resolution, adding a second resolution of the comparison function, and removing the FDPIC register install around a SIGEV_THREAD callback each take the board down with a HardFault rather than printing a FAIL line, which is how a broken callback resolution manifests on Cortex-M.

--fixed-r9 reaches the compiler, checked per the guide, and prints that flag and no other reserved-register flag:

$ make V=1 2>&1 | grep -o -- "--fixed-r[0-9]*" | sort -u
--fixed-r9

tools/checkpatch.sh -f passes on every changed C and header file.

🤖 Generated with Claude Code

https://claude.ai/code/session_01X77FVXXW4bvvBtfFmz1JPz

@github-actions github-actions Bot added Arch: arm Issues related to ARM (32-bit) architecture Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces. Area: BINFMT Board: arm labels Jul 30, 2026
@github-actions

github-actions Bot commented Jul 30, 2026

Copy link
Copy Markdown

MemBrowse Memory Report

hifive1-revb

  • flash: .text +4 B (+0.0%, 83,356 B / 4,194,304 B, total: 2% used)

qemu-armv8a

  • Code: .text.qsort -896 B (+0.0%, 318,202 B)

qemu-intel64

  • Code: .text +16 B (+0.0%, 8,658,064 B)

s698pm-dkit

@casaroli

Copy link
Copy Markdown
Contributor Author
/home/runner/work/nuttx/nuttx/nuttx/tools/fdpic/fdpic-embed: error: execute permissions detected!
/home/runner/work/nuttx/nuttx/nuttx/tools/fdpic/fdpic-verify: error: execute permissions detected!
/home/runner/work/nuttx/nuttx/nuttx/tools/fdpic/nuttx-exports: error: execute permissions detected!

We don't allow executable scripts?

Can anyone please tell me how to proceed?

@acassis

acassis commented Aug 1, 2026

Copy link
Copy Markdown
Contributor
/home/runner/work/nuttx/nuttx/nuttx/tools/fdpic/fdpic-embed: error: execute permissions detected!
/home/runner/work/nuttx/nuttx/nuttx/tools/fdpic/fdpic-verify: error: execute permissions detected!
/home/runner/work/nuttx/nuttx/nuttx/tools/fdpic/nuttx-exports: error: execute permissions detected!

We don't allow executable scripts?

Can anyone please tell me how to proceed?

@simbit18 any idea how to bypass this issue?

acassis
acassis previously approved these changes Aug 1, 2026
DT_INIT_ARRAY, DT_FINI_ARRAY and their size tags are in the base ELF
specification (Figure 5-10) but were missing from the header, which stopped
at DT_BINDNOW.  A loader that wants to run an object's constructors has
nothing to compare d_tag against.

Assisted-by: Claude Code:claude-opus-4-8
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Lets a NOMMU target execute downloadable modules in place from
memory-mapped NOR flash, so a module's text and rodata never consume
RAM.  It is the consumer of xipfs: the loader maps a module's read-only
segment with MAP_XIP_STRICT, which resolves to a direct flash pointer or
fails with -ENXIO rather than falling back to a RAM copy, and pins the
extent for as long as the module is loaded so the defragmenter cannot
relocate code that is executing.

Only the writable segment is copied to RAM, once per running instance.
The loader follows DT_NEEDED so a module can use shared libraries, each
object getting its own GOT and its own data.

FDPIC is what makes this possible: text is position-independent and each
LOAD segment is placed independently, with the text-to-data offset
communicated at load time through function descriptors and the GOT.  A
descriptor is a pair -- entry pointer plus data base -- so a module
function handed back to the firmware carries the data base it needs.
r9 holds that base at runtime, per the ARM FDPIC ABI.

Reserving r9 across the base firmware is what allows a firmware routine
to call back into module code and still arrive with the module's data
base intact.  Toolchain.defs puts --fixed-r9 in ARCHCPUFLAGS rather than
CFLAGS, because almost every board Make.defs assigns CFLAGS with ':='
after including it, which would discard the flag; ARCHCPUFLAGS is
re-expanded by that same assignment and so survives it.  The CONFIG_PIC
--fixed-r10 case is skipped under FDPIC, since reserving both registers
would cost one for nothing.

The DT_NEEDED walk is depth capped.  fdpic_loaddepends() recursed once per
link of a dependency chain with a path buffer on each frame and nothing to
stop it, so a malformed module set overflowed the stack of whichever task
called the loader instead of being rejected.  A dependency *cycle* was never
the hazard -- an object joins the load's list before its own dependencies are
walked, so a library naming something already loaded finds it there and stops
-- what was unbounded is a chain of distinct names, which the list cannot
bound, hence an explicit cap rather than cycle detection.

A module's .rofixup section is skipped, and the file header records why.
.rofixup is the FDPIC self-relocation list a static executable's crt0 walks
to derive its own GOT when no loader is present.  A module links -shared
-nostartfiles, so no crt0 runs, and the built objects hold exactly one entry
there -- the address of the GOT itself, which this loader computes and
installs at every entry into module code anyway.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Two gaps that both fail quietly.

DT_INIT_ARRAY and DT_FINI_ARRAY were ignored entirely.  A C++ module with
any global object therefore loaded, resolved every symbol and ran, with all
of its globals left as .bss reading back zero -- no fault, no log, just
wrong answers.  Both arrays are now walked per object and in dependency
order: an object joins the load's list before the DT_NEEDED walk appends its
own dependencies, so walking that list backwards constructs a library before
the module that needs it, and destruction mirrors it.

Constructors run in whichever task called the loader, before the module's
own task exists, so the FDPIC register does not already hold the object's
data base the way it does once the module is running.  fdpic_callfn()
installs it around each call.  That is safe only because the firmware
reserves the register; being preempted mid-constructor is harmless, since
the register is part of the saved context.

DT_JMPREL was not parsed at all.  Which table an imported function's
descriptor lands in is a linker decision -- -z now puts it in DT_REL, and
without it the same entry goes to DT_JMPREL -- so a module linked the second
way loaded cleanly and then branched to an unrelocated address on its first
call into the firmware.  The symptom is an INVSTATE UsageFault escalated to
a HardFault: no console, no crash dump.  There is no lazy resolver here, so
an unwalked table is not deferred work; both are now bound eagerly, and
nothing is lost by that because a module carries a handful of relocations.

The two tables are not walked identically, which is the part worth
remembering.  In DT_REL the word being overwritten is the addend, and
dropping it breaks a static function reached through its section symbol.  In
DT_JMPREL that same word is the lazy-binding bootstrap -- the address of the
entry's own PLT resolution stub, with a GOT half of -1 -- and adding it to
the resolved symbol value produces an arbitrary address that faults exactly
like the bug this change fixes.  An eager binder overwrites the descriptor
outright.

The descriptor pool is sized from relsize + pltrelsz rather than relsize
alone, so an R_ARM_FUNCDESC in the PLT table cannot run off the end of the
allocation.  GNU ld does not appear to emit that combination -- an
address-taken function is not a call and so never becomes a PLT relocation
-- but the failure it would cause is heap corruption, and the guard is two
additions.

An object declaring RELA PLT relocations is refused rather than misread:
nothing here reads RELA, whose entries are twelve bytes rather than eight.

Assisted-by: Claude Code:claude-opus-4-8
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
A module's function pointer is the address of a two-word descriptor in its
writable segment, not a code address.  Firmware that accepts one, stores it,
and later branches to it therefore jumps into the module's RAM data.  There
is no diagnostic: the board takes a HardFault with a dead console.

qsort and bsearch have resolved the descriptor since FDPIC support landed.
Nothing else did, and nothing noticed, because qsort's comparison function
was the only callback anything exercised.  That left every other entry point
looking perfectly usable -- a module could call one, it would return success,
and the fault arrived later from somewhere else.  Resolution is opt-in per
call site via fdpic_callback(), and this covers the rest of them:
pthread_create, signal, task_create, pthread_once, task_spawn, scandir and
sigaction.

Resolution happens once, in the innermost routine the paths share, for the
reason qsort already documents: resolving twice would treat a code address as
a descriptor.  task_create_with_stack resolves and task_create forwards to
it; nxsig_action() resolves for both signal() and a direct sigaction(), on
the local copy it already makes, so the caller's const struct is untouched
and the SA_SIGINFO form is covered through the union.

Two entry points need care beyond that pattern:

signal() has to exclude the dispositions by hand.  SIG_IGN, SIG_DFL, SIG_HOLD
and SIG_ERR are the integers 0, 1, 2 and -1 rather than addresses, and
fdpic_callback() declines to dereference NULL and nothing else -- handing it
SIG_ERR would read through (void *)-1.

scandir takes two pointers.  Its filter is called by scandir itself and is
resolved here; its comparison function is handed to qsort(), whose own entry
point resolves it, so resolving it here as well would resolve twice.  The
test passes both at once, which is what exercises that distinction.

Resolving the code address is sufficient at all of these.  A thread or task
created from a module inherits its D-Space -- nxtask_dup_dspace() runs before
up_initial_state() installs it in the FDPIC register -- and a signal handler
or pthread_once init routine runs in a context that already holds the
module's data base.

Verified on an RP2350 with a module that hands its own function to each entry
point.  Each assertion was also checked against a deliberate breakage:
removing the scandir filter resolution, and adding a second resolution of the
comparison function, each take the board down with a HardFault rather than
printing a FAIL line -- which is how a broken callback resolution manifests on
Cortex-M, and confirms the assertions test what they name.

mq_notify and timer_create with SIGEV_THREAD remain, and are harder: their
callback runs on a work-queue worker that carries no module data base at all.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
A SIGEV_THREAD notification -- from mq_notify() or timer_create() -- runs its
callback on a shared signal-notification work queue, not in the registering
task.  That worker carries no FDPIC data base, so a module's callback reaches
it with the wrong base and cannot touch its own globals.  This is unlike every
other callback entry point, where the callback runs in a task that inherited
the module's data space and resolving the code address is enough.

The base is knowable exactly once, at registration, when the call is still in
the module's own context: capture it there with fdpic_base() into the
persisted work structure (mq's ntwork, the timer's pt_work).  At send or
expiry the descriptor is resolved to its code address -- a plain memory read
that needs no base -- and stored in work->func.  The worker, seeing a non-zero
base, installs it in the FDPIC register around the call and restores it after;
a zero base, which is every non-module callback, takes the direct path
unchanged.

fdpic_base() exposes the test fdpic_callback() already makes internally --
whether the caller is a module -- for a site that has to decide before it
stores a pointer somewhere the register will no longer be correct.

fdpic_invoke() is the install-call-restore, in the same ARM-thumb inline asm
as the rest of fdpic.h.  It saves the register on the stack and keeps the push
8-byte aligned, and pins the argument in r0, so it asks the allocator for only
two free registers -- enough on builds that also reserve a frame pointer.  It
is safe against preemption: the FDPIC register is REG_PIC in the saved
context, preserved across a context switch, and base firmware reserves it so
no interrupt handler disturbs it.  A context switch or interrupt while the
callback runs therefore keeps the module's base.

Verified with CONFIG_SIG_EVTHREAD on two ARM cores: an RP2350 (Cortex-M33,
armv8-m) on hardware and mps2-an500 (Cortex-M7, armv7e-m) under QEMU.  On each
a module's mq_notify and timer_create SIGEV_THREAD callbacks run on the worker
and write a distinctive value into a module global, proving the base was
installed.  Removing just the register install makes the same callback
HardFault the board, confirming it is load-bearing.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Documentation/components/fdpic.rst covers what someone building or loading an
FDPIC module needs: how the format works and what the loader does with it, how
it differs from NXFLAT and from the ELF loader's own PIC/XIP path, what the
target and toolchain have to provide, how to build a module, a shared library
and a C++ module, and what a firmware entry point must do to accept a module
callback.

The comparison is the part worth stating plainly, because two of the three
properties are shared.  All three formats run position-independent code from
flash with no MMU and give several instances of one module a shared .text with
private .data.  NXFLAT needs its own tools and cannot export symbols, so it
has no shared libraries.  ELF PIC needs no extra tools at all, but one base
register per task means a shared object is loaded as a single allocation --
its text cannot stay in flash -- and there is no DT_NEEDED walk.  FDPIC costs
an arm-uclinuxfdpiceabi linker and buys a pointer that carries its own data
base, which is what makes shared libraries and callbacks on a thread the
module never created possible.

The reference material at the end is the part that is expensive to
rediscover: the FDPIC marker being in EI_OSABI rather than e_flags, the three
relocation types that survive a static link, why both relocation tables are
bound eagerly, why .rofixup is skipped, and the two binfmt contract details
that fail as a panic or a bare -EINVAL.

Also adds the fdpic and reject sections to the xipfs test suite document, which
apps/testing/fs/xipfs grows in the matching apps change.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
@casaroli

casaroli commented Aug 1, 2026

Copy link
Copy Markdown
Contributor Author
/home/runner/work/nuttx/nuttx/nuttx/tools/fdpic/fdpic-embed: error: execute permissions detected!
/home/runner/work/nuttx/nuttx/nuttx/tools/fdpic/fdpic-verify: error: execute permissions detected!
/home/runner/work/nuttx/nuttx/nuttx/tools/fdpic/nuttx-exports: error: execute permissions detected!

We don't allow executable scripts?

Can anyone please tell me how to proceed?

I will rename the scripts to end with the extension.

Modules are built out of tree, and the makefile fragment and scripts that do
it lived in a separate repository until now, which left the loader documented
in this tree but not buildable from it.

``nuttx-fdpic.mk`` reduces a module to three lines of makefile.  The split it
encodes is the whole trick: the stock arm-none-eabi compiler emits correct
FDPIC objects for both C and C++, and only the *link* needs
arm-uclinuxfdpiceabi binutils, so the from-source dependency is binutils
alone.

fdpic-verify checks that a built module's imports resolve against the
firmware's export table, because a module importing a symbol the firmware
does not export links cleanly and fails only once it is on the target, as a
bare -ENOENT that names nothing.  nuttx-exports produces that table from
libs/libc/exec_symtab.c, which has to be run through the preprocessor rather
than read textually.  fdpic-embed turns a built module into a C header, for an
app that has to load a module before there is any way to put files on the
target.

build-binutils.sh builds that one dependency, about a minute.
build-toolchain.sh builds a full FDPIC GCC for anyone who wants one; nothing
here needs it.

BINDNOW is added while moving: the makefile passes -z now by default, and
emptying it leaves imported descriptors in the lazy binding table, which is
the case one of the test fixtures exists to cover.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
The xipfs configuration alongside it stops at the filesystem.  This one
turns on CONFIG_FDPIC as well, so the board runs the module loader and
the FDPIC half of the test suite, and it carries the four options the
loader needs but which are not obviously part of it:

CONFIG_LIBC_EXECFUNCS with a system symbol table, because a module
resolves its imports against one.  POSIX timers stay enabled -- the
callback test module imports timer_create, and with them disabled the
loader cannot resolve that symbol, so the whole module fails to load and
the FDPIC section reports twelve failures that name the callback rather
than the missing timer.  CONFIG_SCHED_HPWORK and CONFIG_SIG_EVTHREAD
carry the SIGEV_THREAD delivery itself.

CONFIG_INIT_STACKSIZE is raised to 16 KB: the test suite runs the
power-loss sweeps from the init task.

CONFIG_NXFLAT is deliberately left out.  The nxflatxip demo's build runs
mknxflat, which the CI container does not have, and that is why
xipfs-nxflat is excluded from the CI build list.  The FDPIC modules are
prebuilt blobs instead, so this configuration builds with a plain
toolchain and CI covers the loader.

Verified on a Pimoroni Pico Plus 2.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Describes what each of the four subcommands shows and why it is worth
showing -- each is one loader property that fails quietly if it is wrong --
and records that the modules are prebuilt blobs because building them needs
a toolchain the tree does not require.

Also enables the example in the board's xipfs-fdpic configuration.

Assisted-by: Claude Code:claude-opus-5
Signed-off-by: Marco Casaroli <marco.casaroli@gmail.com>
Comment thread arch/arm/include/armv7-m/irq.h
#ifdef CONFIG_FDPIC
# define REG_PIC REG_R9
#else
# define REG_PIC REG_R10

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto and armv7-a/armv7-r/armv8-r

# FDPIC reserves r9 above instead; reserving both would cost a register
# for nothing.

ARCHCFLAGS += --fixed-r10

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we switch to r9 for ALL type of PIC

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, doing it — as its own PR ahead of the loader work, because it touches every NXFLAT and PIC board and shouldn't be bisect-tangled with a new binfmt.

The tree already uses both registers, which is the strongest argument for unifying: Toolchain.defs:594 gives CONFIG_BUILD_PIC -mpic-register=r9 while :620 gives CONFIG_PIC r10, twenty-five lines apart, and arm_initialstate.c sets REG_R9 from inline assembly under one and REG_PIC under the other, with the comment reading "Set the PIC base register (probably R10)".

r9 is also the right register rather than an arbitrary one: it is the AAPCS platform register, the static base, and it is what GCC picks for -msingle-pic-base on an EABI target. r10 is the non-EABI default.

And it fixes a combination that cannot build today. Stack checking adds -ffixed-r10 in armv7-m/Toolchain.defs:149 and armv8-m/Toolchain.defs:168 while CONFIG_PIC adds -mpic-register=r10, and GCC rejects the pair with "unable to use 'r10' for PIC register". The comment above REG_PIC has always said the register "can be R9 if stack checking is enabled", but the definition was unconditionally REG_R10, so it would have named the wrong register even had the build succeeded.

One thing worth flagging before that PR appears, because it is not obvious. Switching the register on its own breaks NXFLAT, and silently. The import thunks are generated by mknxflat, which had the register baked into its template as add ip,ip,sl, so a module compiled -mpic-register=r9 reaches its data through r9 while its import stubs still add r10, and it branches to a wild address on its first call into the base firmware. I confirmed this on lm3s6965-ek:qemu-nxflat under QEMU: master passes the nxflat example, the register change alone takes it to a HardFault lockup, and patching the thunk template to r9 restores it exactly.

mknxflat lives outside this repository, in the buildroot NXFLAT toolchain, so the kernel and the tool could drift with no way to detect it — nxflat.h has only h_magic and no version field. So that PR brings mknxflat in-tree first, as tools/nxflat/, with libbfd replaced by reading the ELF symbol table directly. libbfd was never used for anything but opening the file and enumerating symbols, and it is GPL, which we cannot depend on anyway. The thunks are then regenerated from a single NXFLAT_PIC_REG on every build and the two cannot drift.

That import is a no-op on its own: against the upstream tool, for both ARM and Thumb-2, with and without -w, over modules exercising the plain, weak and non-returning thunk paths, the generated thunk fi

ldnxflat does not need to move for this — its only sl refemments — though bringing it in-tree afterwards is worth doing, since it would let us version the format so a stale module fails to load cleanly instead of crashing.

# This goes in ARCHCPUFLAGS rather than CFLAGS because almost every board
# Make.defs assigns
#
# CFLAGS := $(ARCHCFLAGS) $(ARCHOPTIMIZATION) $(ARCHCPUFLAGS) ...

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove the stale comment

Comment thread binfmt/Kconfig
default n
select BINFMT_LOADABLE
select PIC
depends on ARCH_ARM

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why depend on ARCH_ARM, other arch could support FDPIC too

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FDPIC is only specified for ARM (Thumb-2). There is a stale draft specification for RV to support FDPIC but it never got accepted.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread binfmt/fdpic.c
* exists to save, and nothing would report it.
*/

ret = file_ioctl(&loadinfo->file, XIPFSIOC_PIN,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not call mmap, so romfs can work too

Comment thread binfmt/fdpic.c

binfo("fdpic: %s needs %s\n", obj->name, name);

ret = fdpic_loadobject(path, name, head, &dep);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not call dlopen to resolve the so

Comment thread binfmt/fdpic.c
*
****************************************************************************/

static int fdpic_symvalue(FAR struct fdpic_loadinfo_s *obj,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's reuse dlopen, so you can call libelf_findsymbol directly

Comment thread binfmt/fdpic.c Outdated
* is the behaviour a library with state has to have.
*/

struct fdpic_loadinfo_s

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not add FDPIC support on top of the current elf loader

Comment thread binfmt/fdpic.c
* out.
*/

ret = fdpic_runinit(head);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's wrong to call init in the caller context

Comment thread sched/task/task_create.c
****************************************************************************/

#ifndef CONFIG_BUILD_KERNEL
int task_create_with_stack(FAR const char *name, int priority,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we let elf call the base image symbol through PLT? so the PLT could extract function pointer from function descriptor before invoking the real function.

}
#else
# define fdpic_callback(fn) (fn)
# define fdpic_base() (0)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not use up_getpicbase and up_setpicbase

the toolchain has to provide.

========================= ============== ============== ============
Property NXFLAT ELF PIC (XIP) FDPIC

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's better to add FDPIC support into ELF, just like how the origin ELF add the support of PIC(XIP).

Comment thread tools/fdpic/README.md
| `fdpic-verify.sh` | checks a built module's imports resolve against the firmware |
| `nuttx-exports.sh` | turns `libs/libc/exec_symtab.c` into a symbol list |
| `fdpic-embed.py` | turns a built module into a C header, for carrying one in an image |
| `build-binutils.sh` | builds the `arm-uclinuxfdpiceabi` binutils, the one from-source dependency |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not incorporate fdpic special process into the correct ELF file build?

@@ -0,0 +1,221 @@
############################################################################
# tools/fdpic/nuttx-fdpic.mk

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where you support cmake

@casaroli
casaroli marked this pull request as draft August 2, 2026 18:29
@casaroli

casaroli commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

@xiaoxiang781216 I will open another PR so we can compare, because the architectural changes are substantial.

@casaroli

casaroli commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review — it changed the shape of this substantially. Summary of where it's going, then the individual threads below.

The loader merges into ELF. binfmt/fdpic.c goes away and FDPIC becomes a mode of the existing loader. The one structural obstacle is the ET_DYN single load bias, which elf_load.c:580 states outright: "For Dynamic shared objects the relative positions between text and data must be maintained due to references to the GOT. Therefore we cannot do two different allocations." That is exactly what FDPIC repeals. The ET_REL PIC path is already the reference implementation for everything needed — XIP text via xipbase without copying (elf_load.c:590-594 plus the goto skipload at :430), separate data allocation (:615), per-section relocation bases rather than a load bias (elf_bind.c:336-349), read-only sections never patched (:355-360), GOT-based import binding (:346-351), and a fresh dspace_s per exec (binfmt/elf.c:272-285). So this is "make ET_DYN do what ET_REL already does", which is what you meant by "just like how the origin ELF add the support of PIC(XIP)".

DT_NEEDED is dropped entirely. You're right that shared libraries belong to dlopen rather than to a loader-private dependency walker. A module carrying a DT_NEEDED entry will be refused with a clear error rather than loaded and left to fault. That removes fdpic_loaddepends(), the cross-object symbol search, the multi-object list and the owner-GOT rule — about a quarter of the loader, and the quarter with the weakest test coverage. It also removes the only part of the merge that would have changed behaviour for existing ELF and dlopen users, since DT_NEEDED loading does not exist in the tree today; include/elf.h:257 is the tag's only occurrence.

r9 becomes the PIC base register for all of PIC, as a separate PR ahead of this one. That one turned out to be more interesting than expected — details in the register thread.

I'm opening the loader work as a fresh PR rather than force-pushing here, so the two implementations can be compared side by side. I'll close this one once the new one has been looked at.

Two questions still open, in the threads below: the PLT shim, and whether constructors move to the module's context in this series or a follow-up.

@xiaoxiang781216

Copy link
Copy Markdown
Contributor

DT_NEEDED is dropped entirely. You're right that shared libraries belong to dlopen rather than to a loader-private dependency walker. A module carrying a DT_NEEDED entry will be refused with a clear error rather than loaded and left to fault. That removes fdpic_loaddepends(), the cross-object symbol search, the multi-object list and the owner-GOT rule — about a quarter of the loader, and the quarter with the weakest test coverage. It also removes the only part of the merge that would have changed behaviour for existing ELF and dlopen users, since DT_NEEDED loading does not exist in the tree today; include/elf.h:257 is the tag's only occurrence.

it's a great feature, let's do in the new pr by dlopen and bind the needed module during loading.

r9 becomes the PIC base register for all of PIC, as a separate PR ahead of this one. That one turned out to be more interesting than expected — details in the register thread.

I'm opening the loader work as a fresh PR rather than force-pushing here, so the two implementations can be compared side by side. I'll close this one once the new one has been looked at.

Thanks for reorg this patch series.

Two questions still open, in the threads below: the PLT shim, and whether constructors move to the module's context in this series or a follow-up.

it could be done in the new pr after we merge FDPIC.

@casaroli

casaroli commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

DT_NEEDED is dropped entirely...

it's a great feature, let's do in the new pr by dlopen and bind the needed module during loading.

I prefer to bring it back now, because it is easier to test. It is done in the other PR.

@casaroli

casaroli commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #19673

@casaroli casaroli closed this Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Arch: arm Issues related to ARM (32-bit) architecture Area: BINFMT Board: arm Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants