Remy Lebeau

22 hours ago, Dave Nottage said:

I've created a C-based DLL project in C++Builder and am attempting to include the original VS (C-based) code, however I'm receiving a bunch of compiler errors, which are different each time I load the project and hit "Build".

What kind of errors exactly?  Also, which version of C++Builder are you using?  Which of its many C++ compilers specifically are you using?  Are you using a compiler that is compatible with the C++ version that the VS code is targeting?

Quote

Additional issue: I have a .lib file that is used in the VS project, but no corresponding .dll

That would imply that the .lib file is probably a static library instead of an import lib for a DLL.

Quote

Is there a way I can create a .dll in VS so that everything in the .lib is imported?

If the VS project is for a static library rather than a DLL, then compiling it as a DLL may not be helpful since its code would likely not be setup to export symbols that IMPLIB could find.

I think such an HPPEMIT statement should not be needed at all.  Delphi .PAS files output .HPP files, not .H files, when compiled for C++Builder, and there is no need for a .PAS file to ever HPPEMIT a reference to its own .HPP file.

I can confirm the semicolons are required, in ALL C++ versions and compilers.  Typedef statements need to be terminated by a semicolon, just like any other statement in C++.

18 hours ago, Angus Robertson said:

When I add new units to ICS, I have to update literally hundreds of package files for all the old Delphi versions we support.  A few I might fire up that version of that Delphi, but it takes days to do that for all old versions so mostly it's a text editor job. 

Indy has a similar issue, having dozens of packages that have to be updated whenever units are added/removed, and new packages that have to be created when new compilers are released. To help with this task, we created an internal project that has a database of active units and flags to dictate their behavior, and it can generate the various package files, resource files, etc for each compiler version we support. This way, when we do make changes to the units, we simply update the database and regenerate the packages. And when a new compiler is released, we can generate new package files for it.

18 hours ago, Angus Robertson said:

But currently the old C++ packages are untouched, and I suspect they will no longer build anyway, at least not without lots of errors.  So I'll remove all old C++ files from the distribution, and going forward we'll just support 10.2 and later.  If someone needs support for C++ XEx, they will be better working from a newer version than an older version.

Indy works in C++Builder, but doesn't have any C++ project files, only Delphi project files.  For C++, we simply compile the Delphi projects configured with C++ output enabled.

Another alternative for IPC on a single computer is the WM_COPYDATA message, see Using Data Copy.

3 hours ago, Jacek Laskowski said:

from Microsft site:

 

" [Network DDE is no longer supported. Nddeapi.dll is present on Windows Vista, but all function calls return NDDE_NOT_IMPLEMENTED.] "

Network DDE across multiple computers is no longer supported, but Local DDE on a single computer is still supported.

The link that Lars provided is a year old, but the answer is still the same.  Indy does not yet support OpenSSL 1.1+, and so does not yet support TLS 1.3.  It is in the planning stage, no code has been written yet:

https://github.com/IndySockets/Indy/issues/183

39 minutes ago, David Schwartz said:

I'm getting a compiler error from this:

 

[dcc32 Error] RedCon.pas(174): E2250 There is no overloaded version of 'UnicodeFromLocaleChars' that can be called with these arguments
[dcc32 Error] RedCon.pas(175): E2250 There is no overloaded version of 'UnicodeFromLocaleChars' that can be called with these arguments

 

The problem seems to be that the function wants PAnsiChar for Buf, which is an array of AnsiChar.

Typically, a fixed AnsiChar[] array can be passed to a PAnsiChar pointer, but apparently that only works when the low index is 0, whereas yours is 1 instead.  From the documentation:

Quote

An array type of the form array[0..x] of Char is called a zero-based character array. Zero-based character arrays are used to store null-terminated strings and are compatible with PChar values. See "Working with null-terminated strings" in String Types (Delphi).

Quote

EDIT: I changed it to @Buf[1] and that works.

Yes, that will work, and is what you need to do, unless you change your Buf array to use 0-based indexing.

15 minutes ago, chkaufmann said:

I had to set the Accept-Language header, now it works, at least with Russian names.

That is odd, because the TIdHTTP.Request.AcceptLanguage property is empty by default, and according to the HTTP spec:

Quote

A request without any Accept-Language header field implies that the user agent will accept any language in response.

In what way exactly does it not work?  Please show the expected and actual results.  Have you tried sniffing the HTTP traffic to see exactly what is different between TIdHTTP's traffic vs a web browser's traffic?  To see the browser's traffic (since you are using HTTPS), you can use a debugging proxy like Fiddler, or your web browser's own built-in debugger.  To see TIdHTTP's traffic, you can use a debugging proxy, or assign one of Indy's TIdLog... components to the TIdHTTP.Intercept property.

7 hours ago, Incus J said:

To verify the main thread remains responsive I placed a spinning TAniIndicator on the form.  It does spin during the lengthy image processing, however it pauses intermittently a couple of times during the processing of each image, which suggests something is still blocking the main thread?

Then you are probably using Synchronize() ineffectively, or your synced code is doing too much work in the UI thread.  Hard to say without seeing your actual code.

7 hours ago, Incus J said:

Question:  Apart from updating the UI, is there anything else that typically requires a Synchronize?  For example my thread contains file operations - should they be Synchronized too?

ANYTHING that must be serialized with the main thread needs to be synced.  That is USUALLY only the UI, but there may be other things, too.  Again, it really depends on your particular code.  File I/O is not one of them.  Though, if you have multiple threads accessing the same file, they need to coordinate with each other, not with the main UI thread (unless the main UI thread is one of the threads accessing the file).

7 hours ago, Incus J said:

I'm also encountering possible intermittent failure of CreateBitmapFromFile - which works perfectly when I run all my code on the main thread.  So I'm making progress, but would welcome further pointers.

FMX's TBitmap was not thread-safe prior to 10.2 Tokyo, it could not be used safely outside of the main UI thread.  But that was changed in Tokyo, which you say you are using.  Which CreateBitmapFromFile() function are you using exactly?  I don't see any such function in FMX.

27 minutes ago, David Schwartz said:

Thanks, that's great. But it's also why I was puzzled to see the mojibake, because I assumed the data would be converted automatically to Unicode.

Only the string input parameters that are passed directly to CreateProcess() itself are converted, not the data that gets sent over pipes afterwards.

27 minutes ago, David Schwartz said:

I guess the trick is knowing exactly where in the pipeline the conversion needs to occur (if/when it does) and making it happen.

It needs to occur at the point where the data first enters your app, before you pass it up to other areas of your code.  In general, you should use a single encoding inside your app (ie, always use Delphi's native encoding), and then convert data at the last possible moment when it leaves your app, and convert it at the earliest possible moment when it enters your app.

42 minutes ago, David Schwartz said:

To summarize: there's an impedance mismatch of sorts between the command being run and the data it's spitting out vs. the rest of the processing pipeline.

Essentially, yes. 

42 minutes ago, David Schwartz said:

The command I was testing is the simple "dir c:\" command. It's running on Windows 7, so I'm guessing that everything outside this process is all Unicode. Why is a mismatch arising?

Because the "dir" process is outputting its data in ANSI, but the TRedirectedConsole code is expecting the data to be in Unicode when compiled in D2009+.

42 minutes ago, David Schwartz said:

Just out of curiosity, what's the dominating factor here? The app (command.exe); the program (dir c:\); the ReadFile() function; or the type of the storage array used? Or does everything have to be in alignment / agreement? (I did study the code before posting here, and it is a bit mind-numbing in this case.)

CMD.EXE (not COMMAND.EXE) outputs the result of its internal commands (those built into CMD.EXE itself, like "dir") in OEM/ANSI unless you explicitly ask it to output in Unicode via the "/u" parameter. To apply that to "dir" in your example, you would have to run CMD.EXE directly, executing "dir" via the "/c" parameter, then you can add the "/u" parameter, eg:

"cmd.exe /c /u dir c:\"

Note that this only works for internal commands. The "/u" parameter does not effect external processes run from CMD.EXE. They are free to output in whatever codepage they want, or in the calling console's current codepage.

ReadFile() operates on raw bytes only.  Like I said, it has no concept of strings or character encodings. So it will read whatever raw data is sent to the output pipes.  In your situation, "dir" is outputting ANSI and you are reading it into a Unicode buffer, thus the mismatch leading to the mojibake text you see. 

42 minutes ago, David Schwartz said:

What's not quite fully distilled for me is this; it sounds like the "data pipeline" around the CreateProcess call (including all of the various handlers) may need to be tailored for EITHER Ansi OR Unicode. Is this true? Or is there a way to handle both?

Not easily, short of buffering the raw data as-is and then analyzing it to guess what encoding it may be using, and then convert it to the desired encoding if/when needed.

42 minutes ago, David Schwartz said:

I guess it's irrelevant tho, because I plan to use this to run exactly one program (Ora2Pg, which is built on top of a Perl platform). 

That would be an external process outside of CMD.EXE, so you would be subject to whatever character encoding it decides to send its output as.  You will have to test it and see what it sends,and then code for it accordingly.

42 minutes ago, David Schwartz said:

That said, what's the simplest way to deal with this issue?

I think I've already covered that.

The problem is not with CreateProcess[A] itself, but in how the output of the spawned process is [mis]interpreted by TRedirectedConsole when converted to a string format.

What you describe is commonly known as "mojibake", and it can happen when 7/8-bit ANSI character data is mis-interpreted as 16/32-bit Unicode data, thus producing bad "characters" that fall within, in this case, the Chinese language, for instance.

If you look at TRedirectedConsole's internal logic more carefully (specifically, in its ReadHandle() method), you will see that it reads the spawned process's output using the Win32 API ReadFile() function (which has no concept of string characters), stores the raw bytes as-is into a Char[] array, and then copies that array into a native String.  That worked just fine in 2004 when Delphi's native String type was still AnsiString, and the native Char type was still AnsiChar.  But remember that in Delphi 2009, the native String type was changed to UnicodeString, and the native Char type was changed to WideChar.  As such, TRedirectedConsole's current reading logic is no longer valid under a Unicode environment.  It needs to be updated to account for the fact that the 8-bit ANSI encoding of the spawned process's output is now different than the 16-bit Unicode encoding of Delphi's native String type.

One way to handle this would be to tweak TRedirectedConsole to make it explicitly use AnsiString and AnsiChar everywhere, instead of using String and Char, respectively.  This is commonly known as "Ansifying" your code, which is generally shunned upon, but this is a common use case for it.

Also, note that AnsiString is now codepage-aware, so you would have to use the RTL's SetCodePage() function to assign a proper codepage identifier to any AnsiString you give to the user, so the ANSI data will be converted properly when assigned to other strings, such as a UnicodeString when adding to the TMemo (since the entire VCL uses UnicodeString now).  Unless the spawned process is manipulating its output to use a specific encoding (like UTF-8), you can generally use the Win32 API GetACP() function for the codepage identifier, or the RTL's global DefaultSystemCodePage .

For example:

//==============================================================
function TRedirectedConsole.ReadHandle(h: THandle; var s: AnsiString): integer;
//==============================================================
var
  BytesWaiting: DWORD;
  Buf: Array[1..BufSize] of AnsiChar;
  BytesRead: {$IFDEF VER100}Integer{$ELSE}DWORD{$ENDIF};
begin
  Result := 0;
  PeekNamedPipe(h, nil, 0, nil, @BytesWaiting, nil);
  if BytesWaiting > 0 then
  begin
    if BytesWaiting > BufSize then
      BytesWaiting := BufSize;
    ReadFile(h, Buf[1], BytesWaiting, BytesRead, nil);
    SetString(s, Buf, BytesRead);
    {$IFDEF CONDITIONALEXPRESSIONS}
      {$IF CompilerVersion >= 12} // D2009+
    SetCodePage(PRawByteString(@s)^, GetACP(), False);
      {$IFEND}
    {$ENDIF}
    Result := BytesRead;
  end;
end;

...

//==============================================================
procedure TRedirectedConsole.Run( working_dir : string );
//==============================================================
var
    s: AnsiString; // not String!
    ...
begin
    ...
end;

However, other areas of TRedirectedConsole would also have to be ansified, and I can see some issues with doing that, so I would suggest instead to have TRedirectedConsole continue to use the native String type everywhere, and just re-write the ReadHandle() method to explicitly convert the spawned process's output from ANSI to UTF-16, such as with the RTL's UnicodeFromLocaleChars() function (using the same codepage as above):

{$IFDEF CONDITIONALEXPRESSIONS}
  {$IF CompilerVersion >= 12}
    {$DEFINE STRING_IS_UNICODESTRING}
  {$IFEND}
{$ENDIF}

//==============================================================
function TRedirectedConsole.ReadHandle(h: THandle; var s: String): integer;
//==============================================================
var
  BytesWaiting: DWORD;
  Buf: Array[1..BufSize] of AnsiChar;
  BytesRead: {$IFDEF VER100}Integer{$ELSE}DWORD{$ENDIF};
begin
  Result := 0;
  if PeekNamedPipe(h, nil, 0, nil, @BytesWaiting, nil) then
  begin
    if BytesWaiting > 0 then
    begin
      if BytesWaiting > BufSize then
        BytesWaiting := BufSize;
      ReadFile(h, Buf[1], BytesWaiting, BytesRead, nil);
      {$IFDEF STRING_IS_UNICODESTRING}
      SetLength(s, UnicodeFromLocaleChars(GetACP(), 0, Buf, BytesRead, nil, 0));
      UnicodeFromLocaleChars(GetACP(), 0, Buf, BytesRead, PChar(s), Length(s)));
      {$ELSE}
      SetString(s, Buf, BytesRead);
      {$IFEND}
      Result := BytesRead;
    end;
  end;
end;

//==============================================================
procedure TRedirectedConsole.Run( working_dir : string );
//==============================================================
var
    ...
begin
    ...

    {$IFDEF UNICODE}
    // this is important to prevent a crash in CreateProcessW()...
    UniqueString(fCmdLine);
    {$ENDIF}

    // this raises an exception if anything happens
    Win32Check(CreateProcess(nil, PChar(fCmdLine), nil, nil, True,
                NORMAL_PRIORITY_CLASS, nil, wd, fSI, fPI));
    ...
end;

15 hours ago, Tom Mueller said:

With Delphi 10.3 some of these packages cannot be created anymore. Delphi will raise an "Error: E2621 Export table in output file too large: exceeds 65536 limits"

That makes sense, as an executable can have up to 65536 max exported names/ordinals, and up to 65536 max resource strings.  It is a limitation of the EXPORTS and STRINGTABLE formats.

15 hours ago, Tom Mueller said:

There is a bug report about this error https://quality.embarcadero.com/browse/RSP-22698 but it is marked as "Resolved" "Workes As Expected".

That ticket discusses exported names, not string resources.  Two different things.

15 hours ago, Tom Mueller said:

Instead of a package project I tried to include all units into a console application - this compiles without a problem

I would expect it to work for an application, as application's don't usually export very many names.  Resource strings, that is a different matter.

Just how many resource strings are you dealing with?

More information:

Android Developers Blog: Protecting against unintentional regressions to cleartext traffic in your Android apps

Fix Cleartext Traffic Error in Android 9 Pie

android:usesCleartextTraffic

3 hours ago, Dave Nottage said:

What else might it be doing, if the call to the Delphi DLL does not return "properly"?

There are only 2 other ways that I know of:

• a deadlock inside of the function, so it never even tries to exit.
• a corrupted call stack, ie the return address gets overwritten before the function tries to exit.

26 minutes ago, Dave Nottage said:

I still don't know whether or not I have coded the exception handler correctly, or is it just the IntToStr thing that's a problem?

The exception handler itself is coded fine (once you fix the undefined behavior of your IntToStr() implementation), it just may not catch every possible type of exception that can be thrown.  It can only handle SEH exceptions.  But there is no guarantee that what you are experiencing is even related to exceptions.  There may not even be an exception being thrown at all.

On 2/19/2019 at 4:25 PM, Dave Nottage said:

Unfortunately, that is easier said than done. The process which loads the DLL runs only when an app is run on Citrix, and it makes the calls to DriverOpen etc immediately, and once the process fails it needs to be restarted, which makes it nigh on impossible to attach to the process that is loading the DLL, let alone debug it.

I would suggest writing your own test EXE that simply loads the C DLL directly and calls its DriverOpen() function.  Then you can step into the DLL with the debugger to see how it calls into your DelphiDriverOpen() function, see how the parameter data gets passed back and forth, how the call stack is managed, etc.

On 2/19/2019 at 4:25 PM, Dave Nottage said:

For the first point: I'll need to double check that tomorrow, however given that everything works 100% of the time on most machines, I figure it's unlikely.
 

For the second point: in what ways might this happen?

For the third point: That's the reason why I have the exception handler, if I can make it work

• then it may be an environmental issue rather than a coding mistake.  Maybe the offending machine(s) are simply using a different/newer API than what your DLLs use.  Who knows.
• can't answer that seeing the actual code, so stepping through it with a debugger.
• if the exception handler is not being called, then either there is no exception being thrown at all, or it is not an SEH-based exception.

13 hours ago, Dave Nottage said:

I put in an exception handler, but I guess it's not right

No, it is not.  And besides, your IntToStr() has undefined behavior anyway.  You should get rid of that function and just use sprintf() directly in your HandleException().  But that is beside the point.

13 hours ago, Dave Nottage said:

it calls DelphiDriverOpen (which completes in the Delphi DLL), but never reaches the subsequent OutputDebugString

That implies to me that possibly either:

• there is a calling convention mismatch in how DelphiDriverOpen() is declared in both languages.
• the Delphi DLL is corrupting the call stack. 
• an uncaught exception is being thrown.

You will have to use the debugger to verify one way or the other.

You SHOULD already be able to have per-project packages, in all versions of Delphi.  The trick is to install the packages but NOT ENABLE them globally.  With no project loaded, install the packages as needed, and then disable them so they don't appear in the Palettes when no project is loaded.  Then, load a project, enable only the installed packages it actually needs, and close the project.  Repeat as needed for other projects.  Then, from now on, the IDE should load and unload packages on a per-project basis.  At least, this is how it used to work years ago, I haven't tried it in recent years.  Hopefully it still works.

13 hours ago, Dave Nottage said:

A while ago, I managed to make the whole Citrix Virtual Channel thing work, i.e. I constructed a C DLL much like the examples from the Virtual Channel SDK, and just "re-route" the calls to a DLL written in Delphi (long story short: so far it has been the only way I could make this work).

That would imply that the Delphi DLL is not coded properly for the Citrix driver API, since Delphi can certainly produce C-compatible DLLs.

Aside from that, the only thing I see wrong in your Delphi translation is that HND should be a Pointer instead of a THandle, and pLibMgrCallTable should be a Pointer instead of a ^DWORD.

However, you did not show your translation of the PVD type, or of the DriverOpen() and other DLL function declarations.  Since Citrix is complaining about an invalid handle, did you debug your C and Delphi DLLs to log the value of the handle that DriverOpen() actually returns, and the value of the handle that the other DLL functions receive?  Where exactly does DriverOpen() return the handle to - the 'int' return value, or in the PVD parameter?

8 hours ago, CRO_Tomislav said:

I have problem with a next function. How to tranlsate it to a delphi code?

...

I try with:

function MXEIO_E1K_Connect(szIP: String; iPort: Integer; nTimeOut: Integer; handle: Integer; szPassword: String): Integer; external 'MXIO.dll' name 'MXEIO_E1K_Connect';

 

Whene I call this function it crash my app. (Integer overflow).

Is it problem in my sintaks or error in .dll?

Your translation is WAY off.  DONT use String for char* parameters, use PAnsiChar instead.  DONT use Integer for (D)WORD parameters, use (D)WORD instead.  And your handle parameter needs to be declared as 'var' or 'out' since it is an output parameter.

Try this instead:

function MXEIO_E1K_Connect(szIP: PAnsiChar; wPort: WORD; dwTimeOut: DWORD; var hConnection: Integer; szPassword: PAnsiChar): Integer; stdcall; external 'MXIO.dll' name 'MXEIO_E1K_Connect';  

Or this:

function MXEIO_E1K_Connect(szIP: PAnsiChar; wPort: WORD; dwTimeOut: DWORD; hConnection: PInteger; szPassword: PAnsiChar): Integer; stdcall; external 'MXIO.dll' name 'MXEIO_E1K_Connect';

7 hours ago, uligerhardt said:

You didn't specify a calling convention. This is probably cdecl like in MXIO_GetDllVersion. Another poular choice is stdcall.

The DLL's documentation includes VB declarations.  VB doesn't support cdecl at all, only stdcall.

Also, I just found this, which clearly shows the DLL functions using CALLBACK for the calling convention.  That is a preprocessor macro that maps to __stdcall.

6 hours ago, chkaufmann said:

if I enter the following url in my browser, I get a result file with the name "Müller" translated to russian characters:

 

https://translate.googleapis.com/translate_a/single?client=gtx&sl=de&tl=ru&dt=t&q=Müller

 

I tried to implement that with TIdHTTP, but something is missing and I don't find the problem.

 

In general, when creating a URL by hand, I suggest NOT using TIdURI.URLEncode() on the entire URL, but instead use TIdURI.PathEncode() and TIdURI.ParamsEncode() on individual components that actually need to be encoded.

Also, the default encoding for TStringStream is the OS default charset, but JSON typically uses UTF-8 instead.  If you know the actual charset the server uses for the response, you could hard-code it in the TStringStream constructor.  But, it is generally better to just let TIdHTTP handle charset decoding of strings for you instead.

Try something more like this:

var
  http  : TIdHTTP;
  sslIO : TIdSSLIOHandlerSocketOpenSSL;
  url, data  : String;
begin
  http := TIdHTTP.Create(nil);
  try
    http.HTTPOptions := http.HTTPOptions + [hoNoProtocolErrorException, hoWantProtocolErrorContent];

    sslIO := TIdSSLIOHandlerSocketOpenSSL.Create(http);
    sslIO.SSLOptions.SSLVersions := [sslvTLSv1, sslvTLSv1_1, sslvTLSv1_2];
    sslIO.SSLOptions.Mode        := sslmClient;
    sslIO.SSLOptions.VerifyMode  := [];
    sslIO.SSLOptions.VerifyDepth := 0;
    http.IOHandler := sslIO;

    url := 'https://translate.googleapis.com/translate_a/single?client=gtx&sl=de&tl=ru&dt=t&q=' + TIdURI.ParamsEncode('Müller');
    data := http.Get(url);
  finally
    http.Free;
  end;
end;

2 hours ago, Georgge Bakh said:

I'd say in Free Pascal as such tool should be crossplatform and work for FPC too.

FreePascal already has its own Online Package Manager built right into the Lazarus IDE.