system/nxpkg: Make the storage root configurable and crash-safe. by aviralgarg05 · Pull Request #3642 · apache/nuttx-apps

aviralgarg05 · 2026-07-16T07:34:44Z

Note: Please adhere to Contributing Guidelines.

Summary

This PR has been reduced to the storage layer. The rest of what it used to
contain is now in #3718, #3719 and #3720, in that order.

The package store, the catalog and the temporary download area were fixed
paths spread across /etc, /var/lib and /var/cache. A board rarely has
all three, and nothing let it put them somewhere it does have, such as a
removable card. They are now derived from CONFIG_SYSTEM_NXPKG_ROOT.

Every file the package manager owns was also written in place, so a reset
partway through a write left a truncated catalog or database behind. Writes
now go to a temporary file that is renamed into place.

Locking used only the file modification time. A card whose clock does not
follow CLOCK_REALTIME reports a file created a moment ago as decades old,
so a live lock looked stale to the next caller and was taken away from its
owner. Each lock now records a per-boot token and the owning task, so a
contender can tell a live owner from an abandoned one, and reclaims the
lock only in the second case.

Diagnostics go to syslog, since a package operation started by a supervisor
has no console to print to. pkg_runtime_compat() also falls back to
CONFIG_ARCH_BOARD_CUSTOM_NAME, because a board with a custom board
directory defines no CONFIG_ARCH_BOARD at all, which does not build.

Impact

New feature: NO, this is the foundation the other three parts build on.
User adaptation: YES. The storage root moves to
CONFIG_SYSTEM_NXPKG_ROOT, default /var/lib/nxpkg. A board that mounts
writable storage elsewhere sets that symbol.
Build: NO.
Hardware: NO.
Documentation: The companion documentation is Documentation: Update nxpkg and add nxstore guide. nuttx#18875.
Security: NO new boundary.
Compatibility: A store created under the old fixed paths is not migrated;
point CONFIG_SYSTEM_NXPKG_ROOT at the old location to keep it.

Testing

Build host: macOS 26.5, arm64, xtensa-esp-elf-gcc 14.2.0
(esp-14.2.0_20251107).

Target: Xtensa / ESP32-S3, Waveshare ESP32-S3-Touch-LCD-7.

nxstyle, tools/checkpatch.sh, codespell and git diff --check on
every changed file
all system/nxpkg sources compiled for the target
this part builds on master on its own; the three that follow declare it
with Depends-On

On the target: a lock held by a running task was kept while a second caller
asked for it, and locks left by an exited task and by an earlier boot were
both reclaimed.

PR verification Self-Check

This PR introduces one focused change.
I have updated all required description fields above.
I have reviewed and signed every commit.
This PR adheres to the current contribution and coding guidelines.
My PR is still work in progress.
My PR is ready for review and can be safely merged.

aviralgarg05 · 2026-08-06T16:38:10Z

Rebased onto current master (2aebae740) and squashed to a single commit.

The squash also removes an artefact of the previous history: the first commit added system/nxpkg/README.txt and a later commit removed it again after review, which made tools/checkpatch.sh fail on the intermediate commit while trying to resolve a path that no longer exists. All checks pass on the squashed commit.

The diff is limited to system/nxpkg/. Keeping this as a draft until I attach the full HTTP lifecycle transcript (sync, install, update, rollback, remove, and checksum rejection) from the ESP32-S3 target.

linguini1 · 2026-08-11T13:32:56Z

This is a 2.5k LoC patch with 1 commit. Your PR lists a bunch of functionally different features in this PR (i.e adding an icon, adding SHA checks). Please split this into multiple commits, and ideally open a few PRs.

acassis · 2026-08-11T13:38:27Z

This is a 2.5k LoC patch with 1 commit. Your PR lists a bunch of functionally different features in this PR (i.e adding an icon, adding SHA checks). Please split this into multiple commits, and ideally open a few PRs.

I suggest dividing in more commit inside this PR, because he has already many PRs opened and the final evaluation date is arriving

linguini1 · 2026-08-11T13:46:08Z

It won't make much difference except that the patches will be easier to review if split into many.

aviralgarg05 · 2026-08-11T15:32:52Z

It won't make much difference except that the patches will be easier to review if split into many.

I will split and stack the PRs, it will be easy for review then

linguini1 · 2026-08-11T16:02:20Z

Thank you Aviral!

aviralgarg05 · 2026-08-11T19:13:22Z

Split as asked. This PR now carries only the storage layer, and the rest went into three follow-ups:

system/nxpkg: Validate catalog and database contents before use. #3718 catalog and database validation
system/nxpkg: Fetch the catalog and artifacts over the network. #3719 network fetch
system/nxpkg: Complete the package lifecycle and command line. #3720 install lifecycle and command line

Each declares the one before it with Depends-On, so CI applies them in order. Only this one builds on master by itself, which is why it is first.

The content is unchanged from what you already reviewed; it is the same tree, cut into six commits across four PRs.

xiaoxiang781216 · 2026-08-12T06:55:28Z

+ * Private Data
+ ****************************************************************************/
+
+static pthread_once_t g_pkg_lock_boot_once = PTHREAD_ONCE_INIT;


could we remove the global changed variable from nxpkg? so multiple instances could run concurrently.

The package store, the catalog and the temporary download area were fixed paths spread across /etc, /var/lib and /var/cache. A board rarely has all three, and nothing let it put them somewhere it does have, such as a removable card. Derive all of them from CONFIG_SYSTEM_NXPKG_ROOT instead. Also send the diagnostics to syslog, since a package operation started by a supervisor has no console to print to, and fall back to CONFIG_ARCH_BOARD_CUSTOM_NAME when a board defines no CONFIG_ARCH_BOARD, which is otherwise undefined rather than empty and fails to build. Assisted-by: OpenAI Codex:gpt-5.6-sol Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>

Every file the package manager owns was written in place, so a reset partway through left a truncated catalog or database behind. Locking used only the file modification time, which a card whose clock does not follow CLOCK_REALTIME reports as decades old, so a lock created a moment earlier looked stale to the next caller. Write through a temporary file and rename it, and record a per-boot token and the owner task in each lock so a contender can tell a live owner from an abandoned one. Reject a zero-byte write rather than looping on it, and add the path helpers the download and rollback paths need. Assisted-by: OpenAI Codex:gpt-5.6-sol Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>

github-actions Bot added Size: XL Area: System labels Jul 16, 2026

This was referenced Jul 16, 2026

system/nxstore: Add LVGL touchscreen app-store frontend. #3643

Open

games/NXDoom: Add RGB565, module support, and crash fixes. #3644

Open

aviralgarg05 force-pushed the gsoc/nxpkg-sync-lifecycle-hardening-pr5 branch 2 times, most recently from d33ae3b to c4f639c Compare July 17, 2026 19:47

acassis requested review from linguini1, raiden00pl and xiaoxiang781216 July 21, 2026 09:15

acassis reviewed Jul 21, 2026

View reviewed changes

Comment thread system/nxpkg/Kconfig Outdated

linguini1 requested changes Jul 21, 2026

View reviewed changes

Comment thread system/nxpkg/README.txt Outdated