Repository navigation
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new scaling benchmark for @angular/build:library under scripts/benchmarks/library-builder/, including fixture generation, benchmark execution, and integration into the main benchmark CLI. The review feedback highlights several important improvements: resolving Node.js 18 compatibility issues caused by import.meta.dirname, robustly handling broken symlinks to avoid EEXIST errors, validating CLI inputs (sizes, depths, iterations), logging process spawn errors, and wrapping the benchmark execution in a try...finally block to ensure temporary directories are always cleaned up.
dherges
left a comment
dherges
left a comment
There was a problem hiding this comment.
Open questions towards the APF references and benchmark design
|
|
||
| Options (library-builder): | ||
| --layout=<flat|deep> Fixture layout (default: flat) | ||
| --style=<inline|inline-scss|external> Component style variant (default: inline) |
There was a problem hiding this comment.
I think the style variation needs clarification:
inline: a html template embedded in the.tssource fileinline-scss: like inline, with a stylesheet embedded in the.tssource fileexternal: a.scssstylesheet and a.htmltemplate referenced by the.tssource file
My question is: does Angular CLI parse SCSS or just plain CSS from inline styles? Is inlined SCSS even a use case?
The name inline-scss may be confusing. The idea for benchmarking is to have a delta between inline and inline-scss that specifically benchmarks the stylesheet processing.
An alternative is to benchmark inline (embedded template) vs. external (html template file) vs. external-styles (html template file and scss style file). The delta would be baseline vs. baseline + file I/O vs. baseline + file I/O + stylesheet processing.
| } else { | ||
| metadataLines.push(` template: '<div class="ref-comp">Reference component ${id}</div>',`); | ||
| if (style === 'inline-scss') { | ||
| metadataLines.push(` styles: ['.ref-comp { display: block; padding: 4px; }'],`); |
There was a problem hiding this comment.
This is a weak point in the APF references imho. Generating the identical style may lead to superficially high cache hit ratios for the stylesheet processing.
I propose (and will do) randomize the style. To avoid results skewed towards caching everything and have APF references closer to real-world libraries.
| ); | ||
| fs.writeFileSync( | ||
| path.join(entryDir, `${compBase}.component.scss`), | ||
| `.ref-comp {\n display: block;\n padding: 4px;\n}\n`, |
There was a problem hiding this comment.
Same as above.
An option to consider: let's have a delta between plain CSS/SCSS preprocessing. Would we expect vastly different results between the two?
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces a new scaling benchmark subsystem, library-builder, to measure cold build times for @angular/build:library across various layouts and style variants. Feedback on the implementation focuses on improving the robustness of dependency resolution and symlinking in index.mts by replacing manual .pnpm store parsing with createRequire and require.resolve, updating existing symlinks if they point to outdated targets, and preventing a potential crash in parseIntList in benchmark.mts when a single numeric argument is passed.
| import { spawnSync } from 'node:child_process'; | ||
| import fs from 'node:fs'; | ||
| import path from 'node:path'; | ||
| import { performance } from 'node:perf_hooks'; | ||
| import { fileURLToPath } from 'node:url'; | ||
| import { type FixtureOptions, type Layout, type Style, generateFixture } from './fixtures.mts'; |
There was a problem hiding this comment.
Import createRequire to allow robust dependency resolution using Node's standard module resolution algorithm instead of manually parsing the .pnpm store directory.
import { spawnSync } from 'node:child_process';
import fs from 'node:fs';
import { createRequire } from 'node:module';
import path from 'node:path';
import { performance } from 'node:perf_hooks';
import { fileURLToPath } from 'node:url';
import { type FixtureOptions, type Layout, type Style, generateFixture } from './fixtures.mts';
const require = createRequire(import.meta.url);
| if (!fs.existsSync(path.join(rootNodeModules, dep))) { | ||
| linkFromPnpmStore(logger, dep); | ||
| } |
There was a problem hiding this comment.
Update the call site to use the more robust linkDependency helper instead of linkFromPnpmStore.
if (!fs.existsSync(path.join(rootNodeModules, dep))) {
linkDependency(logger, dep);
}
| function linkIfMissing(name: string, target: string): void { | ||
| const linkPath = path.join(rootNodeModules, name); | ||
| try { | ||
| const stat = fs.lstatSync(linkPath); | ||
| if (stat.isSymbolicLink() && !fs.existsSync(linkPath)) { | ||
| // Remove broken symlink to avoid EEXIST error on recreation | ||
| fs.unlinkSync(linkPath); | ||
| } else { | ||
| // Symlink already exists and is valid | ||
| return; | ||
| } | ||
| } catch (e: any) { | ||
| if (e.code !== 'ENOENT') { | ||
| throw e; | ||
| } | ||
| } | ||
| fs.symlinkSync(target, linkPath, 'dir'); | ||
| } |
There was a problem hiding this comment.
If a symlink already exists but points to an outdated or different target (e.g., after dependency updates or switching branches), linkIfMissing currently returns early without updating it. Checking the resolved target of the existing symlink and recreating it if it differs ensures that dependencies are always linked correctly.
function linkIfMissing(name: string, target: string): void {
const linkPath = path.join(rootNodeModules, name);
try {
const stat = fs.lstatSync(linkPath);
if (stat.isSymbolicLink()) {
const resolvedTarget = path.resolve(path.dirname(linkPath), fs.readlinkSync(linkPath));
if (resolvedTarget === path.resolve(target)) {
return;
}
fs.unlinkSync(linkPath);
} else {
fs.rmSync(linkPath, { recursive: true, force: true });
}
} catch (e: any) {
if (e.code !== 'ENOENT') {
throw e;
}
}
fs.symlinkSync(target, linkPath, 'dir');
}
| /** Resolves `dep` (e.g. "rxjs" or "@angular-devkit/core") to its pnpm store dir and symlinks it. */ | ||
| function linkFromPnpmStore(logger: Console, dep: string): void { | ||
| const storeName = dep.startsWith('@') ? dep.replace('/', '+') : dep; | ||
| const candidates = fs.existsSync(pnpmStore) | ||
| ? fs.readdirSync(pnpmStore).filter((entry) => entry.startsWith(`${storeName}@`)) | ||
| : []; | ||
| if (candidates.length === 0) { | ||
| logger.warn(`Warning: could not resolve "${dep}" from the pnpm store; it may be missing.`); | ||
|
|
||
| return; | ||
| } | ||
| // Prefer the lexically-last match (newest version) when more than one is installed. | ||
| candidates.sort(); | ||
| const storeDir = candidates[candidates.length - 1]; | ||
| const target = path.join(pnpmStore, storeDir, 'node_modules', dep); | ||
| if (dep.includes('/')) { | ||
| fs.mkdirSync(path.join(rootNodeModules, path.dirname(dep)), { recursive: true }); | ||
| } | ||
| linkIfMissing(dep, target); | ||
| } |
There was a problem hiding this comment.
Manually parsing the .pnpm store directory is fragile and highly prone to breaking across different pnpm versions, custom store locations, or workspace configurations. Using require.resolve with search paths pointing to the workspace packages, combined with fs.realpathSync, is a much more robust and standard way to locate the physical path of dependencies.
/** Resolves "dep" (e.g. "rxjs" or "@angular-devkit/core") to its physical location and symlinks it. */
function linkDependency(logger: Console, dep: string): void {
try {
const searchPaths = [
path.join(repoRoot, 'packages/angular/build'),
path.join(repoRoot, 'packages/angular_devkit/architect'),
];
const resolved = require.resolve(dep, { paths: searchPaths });
const realPath = fs.realpathSync(resolved);
let target = path.dirname(realPath);
while (target !== repoRoot && target !== path.dirname(target)) {
if (fs.existsSync(path.join(target, 'package.json'))) {
break;
}
target = path.dirname(target);
}
if (dep.includes('/')) {
fs.mkdirSync(path.join(rootNodeModules, path.dirname(dep)), { recursive: true });
}
linkIfMissing(dep, target);
} catch (err) {
logger.warn('Warning: could not resolve "' + dep + '" via require.resolve; it may be missing.');
}
}
1 participant
Measuring that Angular's native library builder scales with grace — far better than ng-packagr: ng-packagr/ng-packagr#3461
PR Checklist
test(@angular/build): ...)behavior change needing its own tests
scripts/benchmarks/library-builder/README.mdPR Type
packages/source changes)What is the current behavior?
Issue Number: N/A
There is currently no way to measure how
@angular/build:library's build time scales with library size (entry-point count, nesting depth, or component style variant).What is the new behavior?
Adds
scripts/benchmarks/library-builder/, a benchmark suite (following the existingscripts/benchmarks/i18n/suite's structure) that generates synthetic APF library fixtures at various sizes and times@angular/build:librarycold builds against them. Run via:Why this is useful
This was built while investigating a performance comparison against ng-packagr (a third-party APF build tool used by most existing Angular libraries today). That investigation found ng-packagr's build time per entry point grows super-linearly once a library exceeds roughly 300-500 secondary entry points -- doubling the library size costs noticeably more than double the build time.
Running the same fixture shapes through
@angular/build:libraryfound no such growth: its per-entry cost keeps falling as the library grows, with no climb even at the largest sizes tested (2000 flat entries / 2048 deeply-nested entries). It was also 1.8-13x faster than ng-packagr at matching sizes, with the gap widening as size increases.@angular/build:library(Deep/nested layout shows the same pattern: 13.0x slower for ng-packagr at the largest size, 2048 entries at tree depth 11.)
This doesn't necessarily reflect anything specific to
@angular/build:library's design beyond "it doesn't have this particular scaling problem" -- the two tools' pipelines are entirely different (different bundler, different incremental-compilation strategy), so this isn't a controlled "same work, different scheduler" comparison. But it is a useful data point: whatever makes ng-packagr's build time climb at scale is apparently not an inherent property of compiling many Angular library entry points, since a different pipeline handles the same fixtures without it.This benchmark suite exists so that:
@angular/build:library's entry-point handling have a regression checkfor this specific scaling characteristic.
Caveats
iterations each) -- enough to establish the shape of the curve, not a high-confidence
statistical comparison.
per-entry-cost curve (grows with size, or doesn't) is the more portable finding across
hardware.
Does this PR introduce a breaking change?
Other information
No changes to any
packages/source in this PR -- only new benchmark tooling and documentation underscripts/benchmarks/library-builder/, plus wiring it into the existingscripts/benchmark.mtsdispatcher as a newlibrary-buildersubsystem alongsidei18n.