perf: improve perf with more delegation - #1090
Conversation
🚀 Deploying Preview to Cloudflare 🚀Preview Deployments by commit
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1090 +/- ##
==========================================
+ Coverage 90.60% 90.70% +0.09%
==========================================
Files 217 220 +3
Lines 20802 21132 +330
Branches 1974 1991 +17
==========================================
+ Hits 18847 19167 +320
- Misses 1948 1958 +10
Partials 7 7 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| File | Main | PR | Change |
|---|---|---|---|
assets/style.css |
18.10 KB | 18.18 KB | +86.00 B (+0.5%) |
Performance estimate (single CI run)
- Generation time: 8.6% faster (35.62 s → 32.56 s)
- Peak memory: 7.6% lower (2.55 GB → 2.36 GB)
legacy-json Generator
Performance estimate (single CI run)
- Generation time: 61.5% faster (21.72 s → 8.37 s)
- Peak memory: 23.3% lower (2.05 GB → 1.57 GB)
llms-txt Generator
Performance estimate (single CI run)
- Generation time: 65.7% faster (21.63 s → 7.42 s)
- Peak memory: 16.9% lower (1.80 GB → 1.50 GB)
orama-db Generator
Output size: 1 file changed · net -4.00 B
File size details
| File | Main | PR | Change |
|---|---|---|---|
orama-db.json |
9.36 MB | 9.36 MB | -4.00 B (-0.0%) |
Performance estimate (single CI run)
- Generation time: 63.3% faster (20.69 s → 7.60 s)
- Peak memory: 23.6% lower (2.04 GB → 1.56 GB)
web Generator
Output size: 2 files changed · net -49.00 B
File size details
| File | Main | PR | Change |
|---|---|---|---|
all.html |
32.46 MB | 32.46 MB | -47.00 B (-0.0%) |
404.html |
21.90 KB | 21.90 KB | -2.00 B (-0.0%) |
Performance estimate (single CI run)
- Generation time: 66.7% faster (130.11 s → 43.29 s)
- Peak memory: 28.5% lower (5.50 GB → 3.94 GB)
Could you expand what these "various performance improvements" are? The PR descriptions should clearly state what the PR does. |
|
@nodejs/platform-riscv64 could you maybe break down this PR after looking at it for a while it feels that it is doing several things at the same time, which makes it harder to understand what is what (ie: what is being changed for performance improv and which piece of that, what is just refactoring) [...] imo even just the piece of threading tuning should be its own PR. Reviewing such large PRs is hard and reduces my ability (and of others) to properly review this PR. |
| const byApi = new Map(pages.map(page => [page.data.api, page])); | ||
|
|
||
| const parts = getSortedHeadNodes( | ||
| pages |
There was a problem hiding this comment.
Could you assign this to its own const, making this line simpler?
| .filter(data => !data.synthetic && !data.chunk && data.api !== 'index') | ||
| ).map(({ api }) => byApi.get(api)); | ||
|
|
||
| const minutes = parts.reduce( |
There was a problem hiding this comment.
I'd argue we shouldn't have a reading minutes on our API docs, only on Learn/Blog content. Nor should we have it on all.html
| ...imports, | ||
| `export const headings = ${JSON.stringify(headings)};`, | ||
| `export const content = () => ${content};`, | ||
| `export default () => renderToStringAsync(<${JSX_IMPORTS.Layout.name} metadata={${JSON.stringify(metadata)}} headings={headings} readingTime={${JSON.stringify(readingTime?.text)}}>{content()}</${JSX_IMPORTS.Layout.name}>);`, |
There was a problem hiding this comment.
Agree, I'd aruge sanitization would be good here
| * @returns {string} | ||
| */ | ||
| export const buildAssetTags = ({ scripts, preloads, stylesheets }, root) => | ||
| [ |
There was a problem hiding this comment.
Could this not be a multi level spread? Also is string manipulation the best way of doing this?
I'd maybe even go further and state that we need to slow down and do this in chunks.
You are doing good good work to reduce memory in pursuit of nodejs/node#62045 but it's hard to keep up with the pace |
|
I've updated the description for more information,
|
4 participants
Fixes #1008
htmlgenerator builds the component library once and the client assetsonce, then compiles, renders, minifies and writes each page on its own in the
worker pool, so memory scales with the largest page rather than with the
site. On the Node.js API docs with section pages, peak memory drops from
about 8 GB to 3.4 GB and the build runs in half the time.
all.htmlis assembled from the module pages' compiled content, in sidebarorder, instead of being built again from every module. The option moved from
jsx-ast.generateAllPagetohtml.generateAllPage, and the page shows noreading time.
jsx-astemits{ data, headings, readingTime, content }per page,contentbeing the page body as one JSX fragment;htmlwraps it in<Layout>.WebBundlercontract is nowbuildServer/compile/buildClient(see the
htmlREADME). Vite plugins no longer see the HTML pages; customizethem through the template, whose
${entrypoint}is replaced by${assets}.Synthetic pages load assets from the site root, so
404.htmlworks at nestedpaths.
@doc-kit/core:getRemarkRehypeWithShikimoved to@doc-kit/core/utils/remark-shiki.mjsandtypeAnnotationToHighlightedHastto
@doc-kit/core/utils/type-annotations/highlighted.mjs, so theastandmetadatastages no longer load Shiki. The defaultthreadsis capped at 4.Function-valued generator configuration (such as a custom
bundler) isdropped before it is sent to workers instead of failing the structured clone.