DW-39: Support TIFF files larger than 2 GB via page-by-page streaming#151
Open
Sawraz-IS wants to merge 1 commit into
Open
DW-39: Support TIFF files larger than 2 GB via page-by-page streaming#151Sawraz-IS wants to merge 1 commit into
Sawraz-IS wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DW-39: Native support for TIFF files larger than 2 GB
Problem
AnyBitmaploaded every file throughFile.ReadAllBytes, which is capped at ~2 GB by .NET's 32-bit array index. TIFF files above that size could not be opened at all — the load failed before any decoding could happen. Customers had to split large TIFFs externally before processing.Change
Add a streaming loader that reads the file with a
FileStreamand decodes one TIFF directory (page) at a time via LibTiff (already referenced), so the whole file is never materialised as a singlebyte[].ReadTiffFrames(Tiff)shared by both the in-memory and streaming paths.AnyBitmap.FromTiffFile(string)streams a TIFF from disk page-by-page.FromFile/ thestringconstructors auto-route TIFF files above the in-memory size limit to the streaming loader, and raise a clear, actionable exception for oversized non-TIFF formats instead of the opaque .NET array-size error.0x2B) in addition to classic TIFF (0x2A), in both little- and big-endian — large multi-gigabyte TIFFs use the BigTIFF layout.The file size is no longer the constraint; only an individual page must fit within one decode buffer.
Validation
FromTiffFile_*tests).Note for reviewers
AnyBitmapmaterialises every frame it holds, so loading all pages of a very large multi-page TIFF at once still scales total RAM with page count × page size. The hard single-buffer (int-indexed array) limit — the actual blocker — is removed; per-page memory stays bounded.