Segmentation Fault in IcsMbToWc

[Sir Alex 1]

Hello.

I'm just tried to convert our application which uses MIME decoder from OverbyteIcsMimeDec.pas

Under Ubuntu I'm got Access Violation (Segmentation fault/Invalid address) in function IcsMbToWc while decoding simple EML file.

 

Problem reproduced by sample code (see below).

 

Sysinfo:

Delphi 11.3 (28.0.47991.2819)

ICS: v8.70 release

Ubunt: 22.04.2 LTS

 

Test code:

program Project1;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils, OverbyteIcsUtils;

var
  TestStr: RawByteString;
  ResStr: string;

begin
  TestStr := 'This is an OpenPGP/MIME signed message (RFC 4880 and 3156)';
  ResStr := AnsiToUnicode(TestStr, CP_UTF8);
end.

Stacktrace (Delphi does not show any stacktrace, so this trace from lldb debugger):

* thread #1, name = 'Project1', stop reason = signal SIGSEGV: invalid address (fault address: 0x7ffffffff000)
  * frame #0: 0x00007ffff7fb16db UTF-16.so`gconv at loop.c:335:7
    frame #1: 0x00007ffff7fb16d0 UTF-16.so`gconv(step=0x00000000007354d8, data=0x0000000000745d10, inptrp=0x00007fffffffe0a8, inend="", outbufstart=0x0000000000000000, irreversible=0x00007fffffffe170, do_flush=0, consume_incomplete=0) at skeleton.c:623:12
    frame #2: 0x00007ffff7d991c2 libc.so.6`__gconv_transform_utf8_internal(step=0x0000000000735470, data=0x0000000000745ce0, inptrp=0x00007fffffffe2f8, inend="", outbufstart=0x0000000000000000, irreversible=0x00007fffffffe170, do_flush=0, consume_incomplete=0) at skeleton.c:675:17
    frame #3: 0x00007ffff7d92a9f libc.so.6`__gconv(cd=0x0000000000745cd0, inbuf=0x00007fffffffe2f8, inbufend="", outbuf=0x00007fffffffe2f0, outbufend=<unavailable>, irreversible=0x00007fffffffe170) at gconv.c:77:13
    frame #4: 0x00007ffff7d9238b libc.so.6`iconv(cd=<unavailable>, inbuf=0x00007fffffffe2f8, inbytesleft=0x00007fffffffe304, outbuf=0x00007fffffffe2f0, outbytesleft=0x00007fffffffe308) at iconv.c:51:16
    frame #5: 0x00000000005a5dd2 Project1`Overbyteicsutils::IcsMbToWc(CodePage=0, Flags=0, MbStr=0x0000000000000000, MbStrLen=0, WStr=Summary Unavailable, WStrLen=0) at OverbyteIcsUtils.pas:2164:1

 

[Angus Robertson 574]

ICS is nor currently supported on Linux, only MacOS.  A new V9 version plans Linux support, but it is a long way from release. 

 

Some of the ICS functions may work on Linux, in OverbyteIcsUtils.pas please note the comment 'Charset conversion functions optionally may use GNU iconv library (LGPL) by explicitly defining conditional "USE_ICONV".  Otherwise ICS attempts to use Windows APIs or just dies.

 

Angus

 

[Sir Alex 1]

Thank you. It's successfully compiled, so I think v8.70 support Linux. :)

 

Looks like USE_ICONV defined for all non windows platforms, so IcsMbToWc function uses libiconv library for conversion, but something going wrong inside iconv function call...

[Angus Robertson 574]

There is a Linux package that allows me to fix non-Windows build issues since I don't have any Apple hardware so not allowed to build for that. 

 

But the low level messaging functions are MacOS specific and simply don't work on Linux, so sockets don't work. 

 

There is an ICS V9 in SVN which does work on Linux, but so far only socket samples, no protocols yet, needs a lot of work.

 

Angus

 

[Fr0sT.Brutal 900]

13 hours ago, Angus Robertson said:

There is an ICS V9 in SVN

Where? I can't see it neither in branches nor in tags 😞

[Angus Robertson 574]

ICS V9 is https://svn.overbyte.be/svn/icsv9/

 

But maybe I've set permissions for private access until more of it is completed. 

 

Angus

 

[Sir Alex 1]

Ok, found a way to solve problem. In fact, System module already have crossplatform versions of functions WideCharToMultibyte and MultiByteToWideChar.

So functions IcsWcToMb and IcsMbToWc can be easily replaced within:

 

{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
function  IcsWcToMb(CodePage: LongWord; Flags: Cardinal; WStr: PWideChar;
  WStrLen: Integer; MbStr: PAnsiChar; MbStrLen: Integer; DefaultChar: PAnsiChar;
  UsedDefaultChar: PLongBool): Integer;
begin
    Result := LocaleCharsFromUnicode(CodePage, Flags, WStr, WStrLen, MbStr,
                                  MbStrLen, DefaultChar, PLongBool(UsedDefaultChar));
end;


{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
function  IcsMbToWc(CodePage: LongWord; Flags: Cardinal; MbStr: PAnsiChar;
  MbStrLen: Integer; WStr: PWideChar; WStrLen: Integer): Integer;
begin
    Result := UnicodeFromLocaleChars(CodePage, Flags, MbStr, MbStrLen, WStr, WStrLen);
end;

 

P.S. Now, MIME decoder works fine

[Fr0sT.Brutal 900]

52 minutes ago, Angus Robertson said:

ICS V9 is https://svn.overbyte.be/svn/icsv9/

 

But maybe I've set permissions for private access until more of it is completed. 

 

Angus

 

Ah, that's it. Thanks, I'll take a look

[Angus Robertson 574]

Quote

System module already have crossplatform versions of functions WideCharToMultibyte and MultiByteToWideChar.

Now perhaps, maybe not 10 years ago when this stuff was written, but I'll bring the unit up to date shortly.

 

Angus

[FPiette 382]

2 hours ago, Angus Robertson said:

Now perhaps, maybe not 10 years ago when this stuff was written, but I'll bring the unit up to date shortly.

Don't forget to check older compilers/RTL because this cross platform is not that old.

[Angus Robertson 574]

LocaleCharsFromUnicode was available in XE2 so is now used from that version.  A lot of conditional code has now gone. 

 

Angus

 

[Fr0sT.Brutal 900]

I believe these functions should be there right from the 1st Unicode version shouldn't they?

Anyway their presence could be checked by $if declared(LocaleCharsFromUnicode)