Hello! 🤓

Summary

When loading large datasets in the base table view, IRFlow Timeline stops being able to navigate past roughly 645k rows even though the import reports the full dataset size correctly.

This is not caused by filtering or SQL row limits. The issue appears in the default base view with no active filters, and it reproduces in both chronological and reverse-chronological sort orders.

Environment

• IRFlow Timeline 1.0.5
• Dataset: output_timeline_stream.csv (cf. https://www.swisstransfer.com/d/b6deef94-69a9-4948-9659-c5245aab6dc5)
• Base timeline view
• No active filters
• No active search
• Default timestamp-based sorting

Reproduction

1. Import a large CSV timeline such as output_timeline_stream.csv.
2. Confirm the application reports the full row count after import.
3. Open the base timeline view with no filters and no search.
4. Scroll through the table in chronological order.
5. Reverse the sort order and scroll again.

Observed behavior

The UI becomes effectively capped at about 645,275 visible/navigable rows.

Example from one affected dataset:

• Full dataset detected: 2,880,571 rows
• Maximum reachable range in the base view: about 645,275 rows

In chronological order:

• First visible row: Row 1 (ID: 3)
• Last reachable row: Row 645275 (ID: 478769)

In reverse chronological order:

• First visible row: Row 645275 (ID: 1881699)
• Last visible row: Row 1 (ID: 2877230)

This strongly suggests the backend is loading and sorting the full dataset, but the renderer cannot represent the full scrollable height.

Expected behavior

The base view should allow navigation across the entire imported dataset, regardless of dataset size, as long as the rows are present in SQLite and no filters are active.

Root cause

The renderer virtualizes the grid by creating a container whose height is:

totalRows * ROW_HEIGHT

and positions each row with:

top = rowIndex * ROW_HEIGHT

For multi-million-row datasets, this produces a virtual DOM height that appears to exceed a practical large-dimension limit in Chromium/Electron's rendering path.

With a row height of 26px, the observed ceiling lines up very closely with a ~16.7Mpx threshold:

• 645,275 * 26 = 16,777,150

This points to a frontend rendering limit rather than a SQL or filtering issue.

On large datasets, the virtualized table ends up creating a scrollable area that is simply too tall for Chromium/Electron to handle reliably, which causes navigation to stop well before the full dataset is reachable.

Proposed fix

Replace the current virtualization strategy with a bounded physical scroll height and a mapping layer between:

• physical scroll position inside a safe DOM height
• logical scroll position across the full dataset

That allows the renderer to keep the scroll container under Chromium limits while still fetching and rendering rows from the full logical row range.

Notes

• This issue affects the main base timeline view, not only the analysis modules.
• The SQL query path already supports full-row counting and paged loading.
• The problem is specifically in the frontend virtual scrolling implementation.

References

• Blink LayoutUnit source: LayoutUnit.h
• WebKit overview: LayoutUnit
• Public repro/discussion of a 16,777,216px Chrome rendering ceiling: Stack Overflow