perf(pm): read each config file once per command via an mtime-validated cache#137
Conversation
…ed cache The pm_engine re-read and re-parsed the same config files several times per command: the root package.json was parsed by aube's parser 3x (apply_config_scope, the unsupported-config scan, the injected-deps default) and by serde 3x (the declared_pm_raw calls in config-scoping, the lifecycle UA, and the install-signals re-read); .yarnrc.yml was opened once per key (immutable, scripts, network, hardened); and the .npmrc ancestor chain was re-walked per key. Add a per-process, mtime+size-validated cache (nub-core config_cache::MtimeCache) and route those readers through it: root_manifest (serde) in nub-core, a shared cached_aube_manifest in pm_engine, and a config-text cache for the yarnrc/npmrc/ bunfig readers in unsupported_config. Each lookup re-stats the file and serves the cached value only when its (mtime, size) stamp is unchanged, so a mid-command rewrite re-reads — the no-stale-read property is structural, not call-ordering- dependent. A missing/unparseable file is never cached, matching the prior read_to_string().ok() behavior exactly. Output-preserving: same parsed values, same command output and exit codes, fewer reads. On a one-command install fixture the reads of each existing config file drop from 3x to 1x.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — a per-process, mtime+size-validated config-read cache that collapses the PM engine's repeated per-command reads/parses of the same config files to a single read each.
- Add
MtimeCache<V>primitive — newcrates/nub-core/src/config_cache.rs: a path-keyedOnceLock<RwLock<HashMap<…, Entry<V>>>>storingArc<V>, validated on a(mtime, size)Stampre-read on every lookup; aNoneread (missing/unparseable) is never cached, and the value is re-stamped after the read. - Route the aube manifest parses through a cache —
AUBE_MANIFEST_CACHE+cached_aube_manifest()inpm_engine/mod.rs, used byapply_config_scope,first_catalog_specifier,manifest_has_injected, andmanifest_has_pnpm_overrides. - Route the unsupported-config readers through a cache —
CONFIG_TEXT_CACHE+read_config_text()inunsupported_config.rsfor the.npmrc/.yarnrc.yml/.yarnrc/bunfig readers (is_ok_and→is_some_andon the classic-yarnrc path). - Cache the root manifest in pin resolution —
ROOT_MANIFEST_CACHEinpm/resolve.rs;root_manifestnow returnsOption<Arc<serde_json::Value>>, resolves the manifest path before keying the cache, and reuses the already-parsedproject.manifeston the no-distinct-workspace-root miss path.
The no-stale-read property is structural — every lookup re-stats and serves the cached value only when the (mtime, size) stamp is unchanged, and the only manifest mutator (write_declared_pm) runs after all config reads. The cache key/content correspondence in root_manifest is sound: detect_project parses root/package.json into project.manifest, and the reuse happens only on the arm where the cache key is exactly that path. The shared statics are test-safe because the unit tests use unique per-test temp dirs, and the config reads run single-threaded before the install fan-out. Unit tests cover hit, mtime invalidation, size invalidation, and not-cached-on-missing.
Claude Opus | 𝕏
|
Shipped in v0.2.1: https://github.com/nubjs/nub/releases/tag/v0.2.1 |

The
pm_enginere-read and re-parsed the same config files several times per command — rootpackage.json(3x via aube's parser, 3x via serde),.yarnrc.yml(once per key), and the.npmrcancestor chain (re-walked per key). This caches each per-invocation so every file is read once.A per-process cache in
nub-core(config_cache::MtimeCache), path-keyed and validated on a(mtime, size)stamp. The serdepackage.json, the aube-parsed manifest, and the yarnrc/npmrc/bunfig readers route through it.Output-preserving with no stale read: every lookup re-stats the file and serves the cached value only when the stamp is unchanged, so a mid-command rewrite re-reads; a missing/unparseable file is never cached, matching the prior
.ok()paths exactly.Verified: reads of each existing config file drop 3x to 1x on an install fixture; cached vs uncached
install/list/why/outdatedgive byte-identical stdout, stderr, and exit codes. Newconfig_cacheunit tests cover hit, mtime invalidation, size invalidation, and not-cached-on-missing.clippy --all-targets --all-features -D warnings,fmt --check, andtest -p nub-cli -p nub-corepass.