fix: garbled display of ISO-2022-JP emails with extended characters

[jo3nzv]

Summary

Japanese emails declaring charset=iso-2022-jp that contain NEC special characters
(e.g. circled digits ①–⑳, Roman numerals Ⅰ–Ⅹ) are displayed as garbled ASCII escape
sequences in SOGo. This PR fixes the issue with a two-stage fallback:

1. Try cp50220 (ISO-2022-JP-MS) via iconv — works on systems with GNU libiconv (e.g. macOS).
2. If that fails (e.g. glibc iconv on Ubuntu/Linux where CP50220 is not available), use a new
   built-in NEC special character decoder (bodyStringFromISO2022JPWithNECExtension) that
   parses the ISO-2022-JP byte stream directly without relying on iconv CP50220 support.

---

Problem

Symptom

An email with Content-Type: text/plain; charset="ISO-2022-JP" and
Content-Transfer-Encoding: 7bit is rendered as garbled text like:

$B$$$D$b$*@$OC$K$J$C$F$*$j$^$9!#(B
$B3t<02q<R%o!<%/%?%s%/$N4X8M$H?=$7$^$9!#(B

instead of the correct Japanese text.

Root Cause

The email claims charset=ISO-2022-JP but its body contains characters from
JIS X 0208 row 13 (NEC special characters: circled digits ①–⑳, Roman numerals Ⅰ–Ⅹ).
This row was introduced by NEC as a proprietary extension and is not defined in
standard ISO-2022-JP (RFC 1468). They are however included in Microsoft's Windows code
page CP50220 (also known as ISO-2022-JP-MS), which is the encoding actually used by
many Japanese Windows email clients.

The following failure chain results:

1. [NSString stringWithData:usingEncodingNamed:@"iso-2022-jp"] is called via
   SOPE's NSString+Encoding which internally invokes iconv.
2. iconv encounters the undefined JIS row 13 and returns EILSEQ
   (illegal byte sequence). The SOPE wrapper returns nil.
3. The nil result satisfies ![bodyString length], triggering the UTF-8 fallback.
4. Because ISO-2022-JP is a 7-bit encoding, every byte in the body is ≤ 0x7F and
   therefore valid UTF-8. The UTF-8 decode succeeds silently, producing a string
   that contains the raw JIS escape sequences as printable ASCII characters
   (e.g. $B, (B). The ESC byte (0x1B) is typically invisible or stripped during
   HTML rendering.
5. The result is the garbled output shown above.

Fix

Before falling back to UTF-8, retry the conversion in two stages:

1. cp50220 (ISO-2022-JP-MS / Windows code page 50220) via iconv. Works on systems
   with GNU libiconv (macOS, etc.).
2. bodyStringFromISO2022JPWithNECExtension — a new built-in decoder added to
   NSData+Mail.m that parses the ISO-2022-JP byte stream directly. It handles JIS
   escape sequences manually, delegates standard JIS pairs to iconv (iso-2022-jp),
   and maps NEC special characters (JIS row 13, first byte 0x2D) to Unicode using a
   small lookup table — with no dependency on CP50220 iconv support. This ensures the
   fix works on Ubuntu 24.04 / glibc where CP50220 is not available.

If both attempts fail the existing fallback chain (UTF-8 → Latin-1) continues
unchanged, so there is no regression.

Changed Files

File	Change
SoObjects/Mailer/NSData+Mail.h	Added declaration for bodyStringFromISO2022JPWithNECExtension
SoObjects/Mailer/NSData+Mail.m	Added cp50220 retry + new bodyStringFromISO2022JPWithNECExtension method in bodyStringFromCharset:
SoObjects/Mailer/SOGoMailObject.m	Added cp50220 + built-in NEC decoder retry in stringForData:partInfo:
ActiveSync/SOGoMailObject+ActiveSync.m	Added cp50220 + built-in NEC decoder retry in three call sites used by the ActiveSync protocol handler

Additional fixes

Fix 2: Standard ISO-2022-JP garbled when SOPE returns pre-decoded NSString (b4a2b73)

When SOPE successfully decodes an ISO-2022-JP body and returns an NSString (instead of NSData), the previous code re-encoded it to UTF-8 bytes and fed them through the NEC decoder — producing garbled output for standard Japanese characters like ■ or ━.

Fix: only attempt Latin-1 re-encoding to recover raw 7-bit bytes. If that fails (the string already contains non-Latin-1 Unicode), use the string as-is.

Fix 3: Standard ISO-2022-JP body not delivered via ActiveSync (b847355)

NSString *s was declared but not initialized to nil. In the NSString branch, when Latin-1 re-encoding failed, if (!s) s = body could silently skip the assignment if s held a garbage (non-nil) value, resulting in an empty/crashed body response and "This message has not been downloaded from the server" on iOS.

Fix: initialize NSString *s = nil.

[Neustradamus]

Dear @Alinto team, @WoodySlum, @c3dr, @ThomasMms23, @QHivert, ...

With the recent big Alinto hack, what is the situation?

• https://www.google.com/search?q=alinto&tbm=nws&tbs=sbd%3A1
• https://cybernews.com/security/alinto-email-data-leak-exposes-traffic/
• https://www.msn.com/en-gb/money/technology/french-email-provider-accidentally-leaked-40-million-records/ar-AA20vQkl
• https://www.scworld.com/brief/misconfiguration-spills-over-40m-smtp-records-linked-to-major-firms

Email provider Alinto accidentally exposed over 40 million SMTP records on a publicly accessible Elasticsearch cluster.
The leak revealed email addresses and traffic metadata from major corporations including L'Oreal, Renault, Carrefour, DHL, and others.
At least 14,000 unique French government email addresses were exposed, including those from embassies, municipalities, and government branches worldwide.
While email content was not leaked, exposed metadata enables targeted phishing attacks and helps attackers map corporate and government communication relationships.

Thanks in advance.

[QHivert]

Hi @jo3nzv, thanks for your PR, I'm aware of it and will take a look as soon as I have time.

Hello @Neustradamus, This leak does not concern SOGo (opensource) nor SOGomail (Alinto product that uses SOGo). The communication has been made directly with the customers impacted.