fix(outlook): read .msg string properties saved in the non-Unicode format

[Guillermo Dols (gdols)]

A .msg saved in the legacy non-Unicode format converts to nothing but scaffolding — # Email Message followed by ## Content, with From, To, Subject and the body all missing. No exception is raised, so the loss is silent.

Why

Every string property in a .msg is stored under a stream whose name ends in the property's MAPI type:

type	meaning	encoding
001F	PT_UNICODE	UTF-16LE
001E	PT_STRING8	the message's 8-bit code page

Outlook writes one or the other for a given message, never both. The converter addressed only the 001F names:

"From": self._get_stream_data(msg, "__substg1.0_0C1F001F"),
"To": self._get_stream_data(msg, "__substg1.0_0E04001F"),
"Subject": self._get_stream_data(msg, "__substg1.0_0037001F"),
...
body = self._get_stream_data(msg, "__substg1.0_1000001F")

so for a non-Unicode message every lookup misses, _get_stream_data returns None for each, and the headers are skipped by the if value: guard while the body is skipped by if body:.

Note this is not reachable by fixing the decode: _get_stream_data never sees any bytes, because the streams it names do not exist in the file.

The change

Each property is now looked up by its tag, trying 001F first and falling back to 001E:

"From": self._get_property_data(msg, "0C1F"),  # PR_SENDER_EMAIL_ADDRESS

PT_STRING8 streams record no encoding of their own, so the charset is detected with charset_normalizer, consistent with how other 8-bit sources are handled in the codebase. Reading PR_INTERNET_CPID out of __properties_version1.0 would give the declared code page instead, but that means parsing the fixed-property stream by hand, and detection is what the rest of the converters already rely on. Happy to switch if you would rather have the declared value.

The Unicode path is untouched — it is still tried first, and still decoded exactly as before.

Tests

packages/markitdown/tests/test_outlook_msg_ansi.py. A .msg is an OLE2 compound file and nothing in the dependency set can write that container, so the streams are served through a stand-in for olefile.OleFileIO rather than a binary fixture:

• a non-Unicode message keeps its headers and body, accents included;
• the same message is not reduced to empty scaffolding;
• a Unicode message converts identically, through the unchanged path;
• the checked-in test_outlook_msg.msg still converts, read through real olefile.

The two non-Unicode tests fail on main and all four pass with this change. The rest of the suite is unaffected and black reports no changes.

Note

This does not overlap #2245, which resolves the Exchange sender DN and the HTML-only body. That PR reads the same 001F stream names, so a non-Unicode message stays empty there.

[Copilot]

🟡 Changes recommended

ANSI streams must use the message’s declared code page to avoid corrupting valid non-Western text.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds support for legacy non-Unicode Outlook .msg string properties.

Changes:

• Falls back from Unicode (001F) to ANSI (001E) property streams.
• Adds ANSI and Unicode regression tests.

File summaries

File	Description
_outlook_msg_converter.py	Reads and decodes ANSI MAPI properties.
test_outlook_msg_ansi.py	Tests ANSI and Unicode message conversion.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

ANSI terminators and several code-page paths can currently corrupt converted text.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

packages/markitdown/src/markitdown/converters/_outlook_msg_converter.py:160

• PR_INTERNET_CPID only declares the encoding of PR_BODY/PR_BODY_HTML, so using it when PR_MESSAGE_CODEPAGE is absent can silently decode ANSI From/To/Subject bytes with the body's unrelated codec (single-byte codecs usually will not raise). Leave header encoding unset in that case so the existing detector handles the undeclared header encoding.

        header_encoding = message_encoding or internet_encoding

• Files reviewed: 2/2 changed files
• Comments generated: 2
• Review effort level: Balanced