Skip to content

DW-39: Support TIFF files larger than 2 GB via page-by-page streaming#151

Open
Sawraz-IS wants to merge 1 commit into
developfrom
DW-39-support-large-tiff-streaming
Open

DW-39: Support TIFF files larger than 2 GB via page-by-page streaming#151
Sawraz-IS wants to merge 1 commit into
developfrom
DW-39-support-large-tiff-streaming

Conversation

@Sawraz-IS

@Sawraz-IS Sawraz-IS commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

DW-39: Native support for TIFF files larger than 2 GB

Problem

AnyBitmap loaded every file through File.ReadAllBytes, which is capped at ~2 GB by .NET's 32-bit array index. TIFF files above that size could not be opened at all — the load failed before any decoding could happen. Customers had to split large TIFFs externally before processing.

Change

Add a streaming loader that reads the file with a FileStream and decodes one TIFF directory (page) at a time via LibTiff (already referenced), so the whole file is never materialised as a single byte[].

  • Refactored the TIFF frame decode into a stream-agnostic ReadTiffFrames(Tiff) shared by both the in-memory and streaming paths.
  • New AnyBitmap.FromTiffFile(string) streams a TIFF from disk page-by-page.
  • FromFile / the string constructors auto-route TIFF files above the in-memory size limit to the streaming loader, and raise a clear, actionable exception for oversized non-TIFF formats instead of the opaque .NET array-size error.
  • Detect BigTIFF (version 43 / 0x2B) in addition to classic TIFF (0x2A), in both little- and big-endian — large multi-gigabyte TIFFs use the BigTIFF layout.
  • Guard against a single page whose decoded pixels would exceed the single-buffer limit.

The file size is no longer the constraint; only an individual page must fit within one decode buffer.

Validation

  • Project builds; 10 TIFF unit tests pass (existing + 3 new FromTiffFile_* tests).
  • Verified end-to-end against a real 3.8 GB / 1200-page BigTIFF: all 1200 pages stream and decode, with peak managed heap ~184 MB (never near the 2 GB wall).

Note for reviewers

AnyBitmap materialises every frame it holds, so loading all pages of a very large multi-page TIFF at once still scales total RAM with page count × page size. The hard single-buffer (int-indexed array) limit — the actual blocker — is removed; per-page memory stays bounded.

@Sawraz-IS Sawraz-IS marked this pull request as ready for review June 15, 2026 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant