Skip to content

Commit 232dcb6

Browse files
committed
docs(devguide): file the open items as proposals, and retire the JIT premise
The pending work surfaced by the full devguide read now has documents where future sessions will look for it, instead of living as notes inside the pages that happened to mention it. New: - opt_in_hover_telemetry.md — stop forwarding hover to the kernel when nobody is listening. The July round deduplicated identical hovers, which fixes a resting mouse and does nothing for a moving one, so the original ~16 messages/s during real hovering remain. Blocked on one decision: what view.hover_target means when telemetry is off. An empty target that looks like "nothing under the cursor" is the silent staleness Contract S7 forbids, so the honest option is an explicit disabled state. - post_1.0/qt_popout_parity.md — the Qt shell is built with include_popout=False, so the whole popup control plane runs only on AnyWidget. Staged at Stage 4 of the host plan, not a defect, but worth naming: the host plan warns about features that work only in standalone and says nothing about the mirror case, which is what R3 was about. - post_1.0/scene_object_owner_field.md — an add-on's shape is indistinguishable from one the user drew. Records why it must not be built by generalising the transient-region mechanism: those hide objects the user should not manage, and an add-on's shape is explicitly theirs to delete. Retired, same day it was drafted: a JIT cold-start proposal. MolSysMT was rewritten in Rust for 1.0 and no longer uses Numba — verified, no .py under molsysmt/ references it and it ships _rust.abi3.so. That removes the premise of standalone_performance_and_depythonization.md: its Path A is moot, its Path B already happened years ahead of its v2.0.0 target, and only Path C survives. The consequence for standalone_v2_evolution_plan.md cuts both ways and both are recorded. Against Option 3: startup latency was its strongest argument, and the Rust half already delivered that without removing Python. For it: the Development Cost row priced a full Rust rewrite that is now done, leaving a wasm32 target and the JS surface. The startup cost of Option 1 has not been measured since the rewrite and must not be quoted from the old figures. pending_proposals/README.md was still directing the reader to finish R2, close D3 and implement D4, all of which landed. It now states that the two implemented proposals are retained deliberately as the design record, which is an exception to its own triage rule and was worth making explicit rather than leaving as an apparent oversight.
1 parent b66f468 commit 232dcb6

7 files changed

Lines changed: 309 additions & 11 deletions

File tree

devguide/benchmarks/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ Measures fresh file loading duration (H5MSM structure files from demo package).
7070
| `1TCD` | 422.61 | 551.85 | 465.83 | 45.98 |
7171
| `chicken_villin_HP35` | 452.68 | 728.14 | 579.79 | 102.57 |
7272

73-
*Note: The high maximum and standard deviation on the `dialanine` loading are caused by the initial python import, dynamic compiler JIT cache load (Numba/MolSysMT), and library warming during the first iteration.*
73+
*Note: The high maximum and standard deviation on the `dialanine` loading are caused by the initial python import and library warming during the first iteration. (This note used to also credit a "dynamic compiler JIT cache load (Numba/MolSysMT)". **MolSysMT was rewritten in Rust for 1.0 and no longer uses Numba**, so that component of the variance no longer exists and these figures predate the change.)*
7474

7575
### 🔄 2. Coordinate Transfer Performance
7676
Measures coordinate extraction (`get_coordinates`) and replacement/scene-rebuild (`set_coordinates`).

devguide/pending_proposals/README.md

Lines changed: 36 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,25 +3,45 @@
33
Only unresolved designs belong here. Implemented plans are promoted to durable
44
documentation or removed; Git retains their development history.
55

6-
## Before 1.0
7-
8-
Two accepted pre-1.0 improvements are active:
6+
## Implemented, retained as the design record
97

108
- [`data_plane_architecture.md`](data_plane_architecture.md): typed buffers for
119
already materialized structural arrays, with behaviorally equivalent JSON
12-
fallback;
10+
fallback. **Closed 2026-07-30.**
1311
- [`runtime_message_router.md`](runtime_message_router.md): one authority and a
1412
typed route across Python, widget/Qt hosts, canvases, and popups.
13+
**Closed 2026-07-30.**
1514

16-
They preserve the current 1.0 scientific model: `view.molsys` remains a complete
17-
selected `molsysmt.MolSys`. The data-plane work removes avoidable
15+
They preserve the 1.0 scientific model: `view.molsys` remains a complete selected
16+
`molsysmt.MolSys`. The data-plane work removed avoidable
1817
`ViewerJSON`/nested-list/text-JSON amplification without introducing partial
1918
residency.
2019

20+
*These two are a deliberate exception to the triage rule "once implemented,
21+
remove the document from this directory."* They are the only written account of
22+
**why** the envelope, the shared manifest and the array-native layout are shaped
23+
the way they are, several documents link to them by path, and the decisions they
24+
record are still load-bearing. Read them as history, not as pending work. If they
25+
are ever moved, the links in `roadmap.md`, `checkpoints.md` and this file move
26+
with them.
27+
2128
Structure windowing, eager/windowed modes, compression, workers, shared memory,
2229
BroadcastChannel, and multiview remain post-1.0. Camera acquisition/movie export
2330
is also explicitly post-1.0.
2431

32+
## Open before 1.0
33+
34+
- [`documentation_execution_in_ci.md`](documentation_execution_in_ci.md): run
35+
`docs/execute_notebooks.py` in CI. Sphinx does not execute notebooks, which is
36+
how ten broken ones survived unnoticed. Postponed by decision, not closed.
37+
- [`opt_in_hover_telemetry.md`](opt_in_hover_telemetry.md): stop forwarding hover
38+
to the kernel when nobody is listening. The July round deduplicated identical
39+
hovers, which fixes a resting mouse and not a moving one. **Blocked on one
40+
product decision** — what `view.hover_target` means when telemetry is off.
41+
*(A JIT cold-start proposal was drafted here on 2026-07-31 and withdrawn the same
42+
day: MolSysMT no longer uses Numba. See
43+
[`../standalone_performance_and_depythonization.md`](../standalone_performance_and_depythonization.md).)*
44+
2545
## Deferred until after 1.0
2646

2747
See [`post_1.0/`](post_1.0/). It contains:
@@ -32,7 +52,16 @@ See [`post_1.0/`](post_1.0/). It contains:
3252
- multiview synchronization;
3353
- advanced annotation, representation, and chemical-metadata work;
3454
- typing-generation and test-output studies;
35-
- deeper large-system rendering analysis.
55+
- deeper large-system rendering analysis;
56+
- [`qt_popout_parity.md`](post_1.0/qt_popout_parity.md): the Qt shell is built
57+
with `include_popout=False`, so the entire popup control plane — manifest
58+
validation, canonical snapshot, endpoint identity — is exercised only on
59+
AnyWidget. Staged at Stage 4 of the host plan, not a defect;
60+
- [`scene_object_owner_field.md`](post_1.0/scene_object_owner_field.md): an
61+
add-on's shape is indistinguishable from one the user drew;
62+
- [`qt_render_check_on_a_gpu_runner.md`](post_1.0/qt_render_check_on_a_gpu_runner.md):
63+
the render gate is closed on real GPU; what is missing is a machine that proves
64+
it again automatically.
3665

3766
These remain useful, but they expand product scope or require benchmark and
3867
upstream decisions. They do not block the current release.
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Opt-in hover telemetry
2+
3+
**Status:** open. Design blocked on one product decision, stated below.
4+
5+
**Origin:** the scalability note in
6+
[`../interaction_targets_and_selection.md`](../interaction_targets_and_selection.md)
7+
§`hover_target`. Promoted to its own proposal on 2026-07-31 because the July
8+
transport round attacked the same problem from a different angle and only solved
9+
half of it — which is exactly the kind of partial fix that gets mistaken for a
10+
closed one.
11+
12+
## The problem
13+
14+
Every debounced hover is forwarded to the kernel. The frontend debounce is
15+
60 ms, so continuous hovering is ~16 messages per second crossing the Comm
16+
channel. On a remote or high-latency deployment (cloud JupyterHub) that competes
17+
with everything else the kernel is doing, and most sessions never read hover
18+
from Python at all — they use it for frontend tooltips.
19+
20+
## What already landed, and why it is not enough
21+
22+
Mol\* re-emits hover on every resolved pick, storing `prevLoci` but never using
23+
it to suppress. A mouse **resting** on one atom therefore produced ~30 identical
24+
messages per second. `registerInteractionObservers`
25+
(`js/src/managers/viewer-controller.ts`) now deduplicates the Python-bound
26+
projection by comparing the serialized payload; local UI still sees every tick.
27+
28+
That fixes the resting mouse completely and the **moving** mouse not at all:
29+
while the pointer travels across the structure every tick is a genuinely
30+
different payload, so nothing is suppressed and the original ~16/s remains.
31+
Deduplication and opt-in are complementary; opt-in is the one that addresses the
32+
case the note was written about.
33+
34+
## The proposal
35+
36+
Forward hover to the kernel only when someone is actually listening:
37+
38+
- a Python callback is registered (`view.on_hover(cb)`), or
39+
- telemetry is explicitly enabled (`view.enable_hover_telemetry = True`).
40+
41+
The frontend is told which state it is in, so the suppression happens **before**
42+
the message is sent, not after it arrives.
43+
44+
## The blocker — decide this first
45+
46+
`view.hover_target` became a public query object *after* the original note was
47+
written, and it is populated **from** the forwarded events. Gating forwarding on
48+
`on_hover(cb)` alone would leave `view.hover_target.info()` silently empty for
49+
anyone who queries it without registering a callback.
50+
51+
That trades a performance problem for a correctness one, and it is precisely the
52+
failure `scene_contracts.md` Contract S7 forbids: an empty target that *looks*
53+
like "nothing under the cursor" is indistinguishable from the truth, and the user
54+
has no way to tell. **A plausible wrong answer is worse than a loud one.**
55+
56+
So the decision to make is: **what does `view.hover_target` mean when telemetry
57+
is off?**
58+
59+
- an explicit "telemetry disabled" state that `info()` reports honestly — the
60+
query still answers, and it answers about itself;
61+
- or `hover_target` counts as a listener, so merely touching it turns telemetry
62+
on for the rest of the session (discoverable, but surprising);
63+
- or telemetry defaults on and the flag is opt-*out* (no correctness risk, no
64+
default gain).
65+
66+
Only the first preserves both properties. It costs one extra state in a public
67+
object.
68+
69+
## Acceptance
70+
71+
- With no callback and telemetry off, hovering sends **zero** hover messages to
72+
the kernel, verified by counting messages at the seam, not by timing.
73+
- `view.hover_target.info()` never reports "nothing hovered" when the truth is
74+
"not being told" — mutation-verified by disabling telemetry and asserting the
75+
reported state changes shape.
76+
- Registering `on_hover` mid-session starts forwarding without a reload.
77+
- The existing deduplication stays: it is what keeps a resting mouse quiet once
78+
telemetry *is* on.
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Popout parity in the Qt standalone host
2+
3+
**Status:** post-1.0, and deliberately staged rather than missing.
4+
5+
**Found:** 2026-07-31, while checking the July popup round against every host.
6+
7+
## What is true today
8+
9+
The Qt shell is built with `include_popout=False`
10+
(`standalone_qt/utils.py:_rebuild_qt_html`), so **no popup window can be opened
11+
in the standalone host at all**. `QtViewChannel.enable_popout` is `False` as
12+
well, though that attribute is inert — Qt bakes its configuration into the shell
13+
HTML at build time.
14+
15+
The consequence is not a broken feature; it is an **untested path**. The whole
16+
popup control plane exists and is exercised only on AnyWidget:
17+
18+
- the `popup_actions` group of `runtime_actions.json` — eleven host/popup actions
19+
with the directions each may carry, validated on both ends;
20+
- `MolSysView.build_popup_scene_snapshot(mode, endpoint)` and the canonical
21+
bootstrap that replaced the replay journal;
22+
- the endpoint identity, token authentication and revocation on close/reopen.
23+
24+
None of it runs in Qt, so none of it is protected there by anything.
25+
26+
## Why this is not a 1.0 gate
27+
28+
`standalone_host_plan.md` §Proposed Execution Stages puts the auxiliary
29+
panel/workbench window at **Stage 4**, explicitly "only after the single-window
30+
host feels solid", and `standalone_direction.md` records the two-screen layout as
31+
"not a near-term implementation target". Roadmap gate 4 asks for real-window
32+
Qt/WebGL validation of load, interaction, context menu and the live-replacement
33+
regression — it does not ask for popout.
34+
35+
`standalone_v2_evolution_plan.md` Phase 1 already lists implementing the
36+
`createWindow` handler as an action item. This document exists so the *current*
37+
asymmetry is recorded where the popup work can find it, not to reopen the
38+
staging decision.
39+
40+
## The asymmetry worth naming
41+
42+
`standalone_host_plan.md` §Non-Negotiable Invariants warns about features that
43+
work **only in standalone** because the host invented viewer logic. It says
44+
nothing about the mirror case: a feature that works in the notebook and not in
45+
standalone. That mirror is exactly what R3 was about — an unknown action was
46+
rejected observably on AnyWidget and accepted in silence on Qt — and popout is
47+
the same shape at the level of windows rather than actions.
48+
49+
The invariant list should be read as symmetric: **one workbench model, multiple
50+
hosts** is violated in both directions.
51+
52+
## What it needs
53+
54+
1. A `createWindow` handler on the Qt page, mapping the request to a real
55+
`QWebEngineView` in its own window.
56+
2. Building the shell with `include_popout=True` for the host that has such a
57+
handler, and keeping it `False` where there is nowhere for a window to go.
58+
3. The popup channel's identity handshake over the Qt transport — the token and
59+
endpoint model is transport-agnostic by construction, but that is an
60+
assertion until something runs it.
61+
62+
## Acceptance
63+
64+
- A canvas popout opens from the Qt standalone and receives the canonical
65+
snapshot, with the same scene the host canvas shows.
66+
- A panel popout opens and every Studio subpanel is populated — the Contract S1
67+
failure (a blank section, because a runtime-only summary was not projected) is
68+
host-independent and must be checked here too.
69+
- The popup-channel E2E, or its Qt equivalent, runs against the real host rather
70+
than only against Chromium.
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# An `owner` field on scene objects
2+
3+
**Status:** post-1.0. Deferred on purpose, recorded so it stays a choice.
4+
5+
**Origin:** `scene_contracts.md` §0.12, which decided the behaviour and deferred
6+
the field in the same breath.
7+
8+
## The situation
9+
10+
Add-ons do not have a private channel. They create shapes by calling the public
11+
API — ElastNetMT does exactly this:
12+
13+
```python
14+
layer = view.shapes.add_displacement_vectors(...) # adapters/modes.py
15+
layer = view.shapes.add_links(...) # adapters/contacts.py
16+
```
17+
18+
So an add-on's shape lands in `_scene_objects` like any other, appears in
19+
`shapes.info()`, and shows up in the Shapes panel **with a trash button next to
20+
it**.
21+
22+
**That was decided to be correct and it stays.** It is the user's scene; a viewer
23+
that displays an object it refuses to let you remove is worse than one that lets
24+
you delete something an add-on made. The add-on's obligation is to tolerate it:
25+
the handle it kept goes `_active = False`, and it must check rather than assume.
26+
27+
## What is missing
28+
29+
There is no `owner` field in the model, so a shape produced by ElastNetMT is
30+
**indistinguishable** from one the user drew by hand. The panel cannot say
31+
`· from elastnetmt`, and the user has no way to tell which objects will
32+
reappear if the add-on re-runs and which are theirs.
33+
34+
## Why it was deferred
35+
36+
It is new public API surface, and the block that surfaced it already carried
37+
enough. That reasoning still holds; this is not urgent.
38+
39+
## What it would take
40+
41+
- an `owner` on the scene-object record, set at creation from the calling add-on
42+
when there is one, `None` otherwise;
43+
- the field in the authoritative summary (Contract S1) so the panel can render it
44+
without a second projection;
45+
- serialisation (Contract S5), or an explicit decision that ownership is a
46+
session-scoped annotation and does not survive a reload — either is defensible,
47+
but it must be *decided*, because an owner that silently disappears on import
48+
is worse than no owner at all;
49+
- a row affordance that is informative, not a second permission system. Ownership
50+
labels what made an object; it does not restrict what the user may do to it.
51+
52+
## Related, and deliberately not the same thing
53+
54+
Transient regions (`focus`, `orientation`, `plane`) are already filtered from the
55+
panel and from `export_state` by `_TRANSIENT_REGION_TAG`, and Contract V's
56+
*owned primitives* — shapes a measurement creates for its own realisation — must
57+
be filtered the same way. Both are about objects the user did not create and
58+
should not manage directly.
59+
60+
`owner` is different: an add-on's shape **is** the user's to manage. Do not
61+
implement `owner` by generalising the transient mechanism; they answer opposite
62+
questions.

devguide/standalone_performance_and_depythonization.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,32 @@
11
# Standalone Performance and De-pythonization Roadmap
22

3+
> **Overtaken by events (2026-07-31). Read the rest of this document as the
4+
> analysis that led to a decision, not as pending work.**
5+
>
6+
> **MolSysMT no longer uses Numba. It was rewritten in Rust for 1.0.** Verified
7+
> against the installed package: zero `.py` files under `molsysmt/` reference
8+
> `numba`, and it ships a compiled `_rust.abi3.so`.
9+
>
10+
> That means **Path B / Phase 2 has already happened**, years ahead of the
11+
> v2.0.0 target this roadmap assigned it — and it happened in MolSysMT rather
12+
> than as a separate `molsys_core` crate. The consequences for this document:
13+
>
14+
> | Section | Status |
15+
> |---|---|
16+
> | §1 "The Cold Start Problem" — the 3–5 s JIT freeze | **Gone.** The premise no longer exists. |
17+
> | Path A (Numba cache / AOT) | **Moot.** There are no `@jit` decorators left to cache. |
18+
> | Path B (Rust + PyO3) | **Done**, in MolSysMT itself. |
19+
> | Path C (WASM / de-pythonization) | **Still open**, and now much closer: the Rust core it presupposed exists. Its own preconditions — API parity between the Python library and a WASM/JS surface, and a Tauri-class shell — are untouched. |
20+
> | Phase 1 of the Recommended Roadmap | Withdrawn. |
21+
>
22+
> What is worth re-deriving rather than assumed: the standalone host's *actual*
23+
> startup cost today. This document's numbers all measured JIT compilation. Any
24+
> remaining startup latency now has a different cause — import cost, Qt WebEngine
25+
> initialization, or first-load serialization — and none of it has been measured
26+
> since the rewrite. **Do not quote the 3–5 seconds below as current.**
27+
>
28+
> Path C's evaluation stands and is the only part of this file still live.
29+
330
This document outlines the strategic roadmap for addressing the JIT compilation latency ("cold start" problem) in the `molsysmt` / `molsysviewer` backend, particularly for non-programmer desktop environments. It analyzes immediate workarounds and long-term architectural transitions to Rust and WebAssembly (WASM).
431

532
---

0 commit comments

Comments
 (0)