aehimself

Calling UniqueString(_bdspath) before the CreateProcess call still produces an error. Maybe it's even unnecessary...?

https://www.oreilly.com/library/view/delphi-in-a/1565926595/re354.html

"If you cast a string to PChar, Delphi also ensures that the PChar points to a unique instance of the string. If you modify the string through the PChar pointer, the other references to the original string are safe."

I don't know, I am in completely uncharted territory here. Never had to think more of PChar castings than if a method needed it instead of a String.

18 minutes ago, David Heffernan said:

First argument of CreateProcess must be writeable? Is it? 

I was unaware of this, and even MSDN specifies it as IN only. Do you maybe have some references I can look into?

Changing it to

  pc: PChar;
Begin
  [...]

  GetMem(pc, Length(_bdspath) + 1);
  Try
    StrPCopy(pc, _bdspath);

    If Not CreateProcess(pc, nil, nil, nil, False, CREATE_NEW_PROCESS_GROUP, nil, nil, StartInfo, ProcInfo) Then
      RaiseLastOSError;
  Finally
    FreeMem(pc);
  End;

still throws the AV.

I am really baffled by the nature of this issue. Popping up at window messages, and being "execution at address" I'd say it's like a memory corruption. I have doubts though as commenting out the two CloseHandle lines will delay / get rid of these. That clearly points to improper use of CreateProcess...

I'm always using a "subscriber model" when it comes to logging. I have a singleton class, with 2 important methods: Subscribe and Log. You can subscribe with a

TLogProc = Procedure(Const inLineToLog: String) Of Object;

for example, which is only adding the method pointer to a TArray / TList.

Logging uses a critical section and loops through all subscribed loggers, like:

Procedure TMyLogger.Log(const inLineToLog: String);
Var
 logrefproc: TLogProc;
 s: String;
Begin
 TMonitor.Enter(Self);
 Try
  s := '[' + DateTimeToStr(Now)+ '] ' + inLineToLog.Trim;

  For logrefproc In _logrefprocs Do
   Try
    logrefproc(s);
   Except
    // Swallow logger exceptions.
   End;
 Finally
  TMonitor.Exit(Self);
 End;
End;

This way you can call MyLogger.Log from any thread and the message will be distributed to all subscribed loggers. This way, printing to the console window, appending to a TMemo, saving it to a local file and pusing it up to the cloud can all happen - if you wish. As you already seem having some kind of a logger object, moving it to a separate unit and using this logic should suit your needs well. Just make sure the logging methods aren't taking long AND are synchronized, as they will be called by the thread's context! When you have lots of entries I found it easier to add them to a TList<String> object and using a TTimer to dump the contents and empty the list every second or so:

Procedure TLogFrame.InternalLog(const inLogWhat: String);
Begin
 System.TMonitor.Enter(_linestolog);
 Try
  _linestolog.Append(inLogWhat + sLineBreak);
 Finally
  System.TMonitor.Exit(_linestolog);
 End;
End;

Procedure TLogFrame.LogTimerTimer(Sender: TObject);
Begin
 LogTimer.Enabled := False;
 Try
  System.TMonitor.Enter(_linestolog);
  Try
   If _linestolog.Length > 0 Then
   Begin
    LogMemo.Lines.BeginUpdate;
    Try
     LogMemo.Text := LogMemo.Text + _linestolog.ToString;

     CursorToEnd;
     _linestolog.Clear;
    Finally
     LogMemo.Lines.EndUpdate;
    End;
   End;
  Finally
   System.TMonitor.Exit(_linestolog);
  End;
 Finally
  LogTimer.Enabled := True;
 End;
End;

And my OCD forces me to tell you, that

On 12/6/2022 at 2:28 PM, felixeckert said:

 

  if uLogAbstractor.loglvl = LOG_LVL_DBG then
  begin
    if FLogger.FLogDebug then
      memoLog.Lines.Add(FLogger.FLastMsg);
  end else
    memoLog.Lines.Add(FLogger.FLastMsg);

 

can be simplified to:

  if (uLogAbstractor.loglvl <> LOG_LVL_DBG) Or FLogger.FLogDebug then
      memoLog.Lines.Add(FLogger.FLastMsg);

After solving some mysteries in DDE I decided to extend my Delphi instance helper with a new method: to launch a new Delphi instance of a selected version. The theory is simple: launch bds.exe and return the TDelphiInstance object only when the IDE is available (this was removed to further simplify the calls happening).

While the new process starts, I'm getting Access violation at address 00000000 in module 'Project3.exe'. Execution of address 00000000 upon:

- Calling ShowMessage (WinApi.CommCtl : 13107, TaskDialogIndirect)

        Result := _TaskDialogIndirect(pTaskConfig, pnButton, pnRadioButton,
          pfVerificationFlagChecked)

- Application.ProcessMessages loop (Vcl.Forms : 11460, TApplication.ProcessMessage)

            DispatchMessageW(Msg)

- Closing the instance which was created by CreateProcess (error raises in ProcessMessages, just like above)

From the third appearance I believe I'm not using CreateProcess correctly (in theory, after closing all handles my application should have nothing to do with the instance created) but if I move the code out of TBorlandDelphiVersion to the project itself, it just works.

Launching the exe is performed as follows:

Function TBorlandDelphiVersion.NewDelphiInstance: TDelphiInstance;
Var
  StartInfo  : TStartupInfo;
  ProcInfo   : TProcessInformation;
  ec: Cardinal;
Begin
  Result := nil;

  FillChar(StartInfo, SizeOf(TStartupInfo), #0);
  StartInfo.cb := SizeOf(TStartupInfo);
  FillChar(ProcInfo, SizeOf(TProcessInformation), #0);

  If Not CreateProcess(PChar(_bdspath), nil, nil, nil, False, CREATE_NEW_PROCESS_GROUP, nil, nil, StartInfo, ProcInfo) Then
    RaiseLastOSError;

  Try
//    If Not GetExitCodeProcess(procinfo.hProcess, ec) Then
//      RaiseLastOSError;
//
//    WaitForInputIdle(procInfo.hProcess, INFINITE);

    Repeat
      If Not Assigned(Result) Then
      Begin
        Self.RefreshInstances;
        Result := Self.InstanceByPID(procinfo.dwProcessId);
      End;
    Until Assigned(Result);// And ((Result.IDEHWND <> 0) Or Result.FindIdeWindow) And Not Result.IsIDEBusy; // Normally here, removed for simplicity
  Finally
    CloseHandle(ProcInfo.hThread);
    CloseHandle(ProcInfo.hProcess);
  End;
End;

I attached the full source of said class, you can reproduce the issue by...

Uses AE.DelphiVersions;

procedure TForm3.Button1Click(Sender: TObject);
begin
 ShowMessage(ni.ToString);
End;

procedure TForm3.Button2Click(Sender: TObject);
Var
 dv: TDelphiVersions;
 c: Cardinal;
begin
 dv := TDelphiVersions.Create(nil);
 Try
  c := dv.LatestVersion.NewDelphiInstance.PID;
//  Self.Caption := c.ToString; // This works, no exception is raised
  ShowMessage(c.ToString);
 Finally
   FreeAndNil(dv);
 End;
end;

function TForm3.ni: cardinal;
Var
 dv: TDelphiVersions;
begin
 dv := TDelphiVersions.Create(nil);
 Try
  Result := dv.LatestVersion.NewDelphiInstance.PID;
 Finally
//   FreeAndNil(dv); // If you do not free this up, exception is raised in ShowMessage.
 End;
end;

MadExcept reports no buffer under- or overruns, no memory, handle or any other leaks are reported by DeLeaker.
 

I never really used CreateProcess, and never actually managed to produce an execute at address AV up until this point. Can anyone point me to the right direction?

P.s.: Using Delphi 11.2, compiled app is 32 bit, cannot check if the issue will appear anywhere else.

AE.DelphiVersions.zip

Without seeing the code it's really hard to tell what is eating up the memory. I also have zero experience with FireDac, but...

I suspect you have a query, with a DBGrid connected and this query also downloads the BLOB. The issue is that the query will at least download all the blobs for all records which are visible in the grid, worst case it downloads all. This is already using up a huge amount of memory. When you assign the value to a string, you allocate even more memory, resulting in OOM.

What I'd try is to exclude the blob from the main query, add a second query with SELECT blobfield FROM MyTable WHERE ID = x. You can open this manually upon record change, load the data in the RichEdit and close it immediately. That should minimize the memory usage.

https://github.com/aehimself/AEFramework/blob/master/AE.DelphiVersions.pas finally updated, if anyone will need this in the future.

I'll attempt the reporting sometime next week. Never did that before 🙂

@Attila Kovacs

// Need to reimport these two

function UnpackDDElParam(msg: UINT; lParam: LPARAM; puiLo, puiHi: PUINT_PTR): BOOL; stdcall; external user32;
function FreeDDElParam(msg: UINT; lParam: LPARAM): BOOL; stdcall; external user32;

// class procedure TDdeHelper.DdeWndProc
  pLo, pHi: PUINT_PTR; // NOT IntPtr

  GlobalUnlock(pHi^);
  GlobalFree(pHi^);

And your code should be 64-bit compatible 🙂

I'll run some more tests but it seems that now it is finally working.

It's not a threading issue, it's 64-bit issue!

According to MSDN, it's declared as:

BOOL UnpackDDElParam(
  [in]  UINT      msg,
  [in]  LPARAM    lParam,
  [out] PUINT_PTR puiLo,
  [out] PUINT_PTR puiHi
);

In Delphi, however:

Now, we are feeding it an LParam, which is NativeInt.

Time to re-import it, correctly this time 🙂

Your original idea was good, I'm indeed sending WM_DDE_INITIATE first. However attempts to purge the queue is unsuccessful, as there is nothing in the message queue:

  msghwnd := AllocateHWnd(nil);
  Try
    atomservice := GlobalAddAtom(PChar(_service));
    atomtopic := GlobalAddAtom(PChar(_topic));
    Try
      SendMessage(inDDEServerHWND, WM_DDE_INITIATE, msghwnd, Makelong(atomservice, atomtopic));
    Finally
      GlobalDeleteAtom(atomservice);
      GlobalDeleteAtom(atomtopic);
    End;

    While PeekMessage(msg, msghwnd, 0, 0, PM_REMOVE) Do
     Begin
       Sleep(20);
     End;

I split LParam by LoWord and HiWord, in that case FreeDDElParam(msg.message, msg.lParam) throws an AV...

it's like LParam is malformed somehow...

...any idea why this throws an AV? DDE is initialized inside the thread's context, I think the PeekMessage's section runs in the thread's context too...

The message goes through though, file is opened in the IDE, msg.WParam indeed contains the handle of the DDE server, msg.hwnd is equal to the handle of the window created in the beginning... I don't know what else to check 😞

  msghwnd := AllocateHWnd(nil);
  Try
    [...]

    PostMessage(inDDEServerHWND, WM_DDE_EXECUTE, msghwnd, commandhandle);
    SetTimer(msghwnd, 1, inTimeOutInMs, nil);

    Repeat
      If PeekMessage(msg, 0, 0, 0, PM_REMOVE) Then
      Begin
        If msg.message = WM_TIMER Then
          Break;

        If msg.message = WM_DDE_ACK Then
        Begin
          If UnpackDDElParam(msg.message, msg.lParam, @pLo, @pHi) Then // AV is thrown on this line
          Begin
            GlobalUnlock(pHi);
            GlobalFree(pHi);
            FreeDDElParam(msg.message, msg.lParam);

            PostMessage(msg.wParam, WM_DDE_TERMINATE, msghwnd, 0);

            Exit;
          End;
        End;

        TranslateMessage(msg);
        DispatchMessage(msg);
      End;

      Sleep(200);
      Inc(wait, 200);
    Until wait >= inTimeOutInMs;
Finally
 DeallocateHWnd(msghwnd);
End;

Okay, this version works! Probably the delayed freeup of resources, in my case when the message arrived the handles were already invalid.

Now, just to get rid of the GetMessage loop, as it breaks the ability to be run in a thread. Or I need a message pump 🙂

I made a discovery...

At the moment there are 3 possible outcomes:

- You start it from the IDE, works. Period.

- You start it from outside of the IDE, WHILE the IDE is running -> Getting the lust succeeds but file is not opened:

- You start it from outside of the IDE, and NO IDE is running -> 16349.

This makes me believe that 16349 should be handled separately, it means there is no list to be gathered... which equals to no Delphi instances running. Does this make sense...?

  If HWND(inMessage.WParam) = _ddehwnd Then
    If UnpackDDElParam(inMessage.Msg, inMessage.lParam, @pLo, @pHi) Then
    Begin
      GlobalUnlock(pHi);
      GlobalFree(pHi);
      FreeDDElParam(inMessage.Msg, inMessage.lParam);
    End;

The innermost section never gets executed, so yes, it indeed stops the AV from happening 🙂 Now only unable to connect from outside the IDE remains.

I'm still trying to find that article.

33 minutes ago, Attila Kovacs said:

The whole DDE fuck is just some botching from MS. I'm sure they can't even document it as they don't know why is it working.

😄

It only fails on one machine, on mine it works. Maybe it's an AV-related thing...

Edit: DDE version locks up the IDE completely if ran on the machine which is affected, also throws an AV here after the WM_DDE_INITIATE message is sent:

When starting it outside from the IDE, reports the same DDE error:

I also can confirm, running my program on my machine outside of the IDE also fails to join the conversation.

Crap, I remember reading something about DDE behaving differently from the IDE somewhere...

So basically the difference is the cleanup in the message handler? Afaik ackINIT does nothing, as the window is not even alive in that section.

Question, why are you reading back out the partner and service name? In theory they will always come back as you specified in DdeConnectList because you are defining both parameters.

Edit: first test, after PC restart, no IDE was opened yet:

😄

My code is your initial version, I'm not processing anything 🙂

  msghwnd := AllocateHwnd(nil);
  Try
    atomservice := GlobalAddAtom(PChar(DDESERVICE));
    atomtopic := GlobalAddAtom(PChar(DDETOPIC));
    Try
      SendMessage(_ddehwnd, WM_DDE_INITIATE, msghwnd, Makelong(atomservice, atomtopic));
    Finally
      GlobalDeleteAtom(atomservice);
      GlobalDeleteAtom(atomtopic);
    End;

    cmd := '[open("' + inFileName + '")]';
    commandhandle := GlobalLockString(cmd, GMEM_DDESHARE);
    Try
      PostMessage(_ddehwnd, WM_DDE_EXECUTE, msghwnd, commandhandle);
    Finally
      GlobalUnlock(commandhandle);
      GlobalFree(commandhandle);
    End;
  Finally
    DeAllocateHwnd(msghwnd);
  End;

Maybe I need to switch up that PostMessage to SendMessage. I'll give that modification a spin.

Hehe, you are right. Just this moment I entered a locked-up state again. 16394 until restarting the IDE. Starting the application from within our outside the IDE makes no difference. Double-clicking a file in explorer indeed does open the file in Delphi.

It's strange but I THINK it started with the usual command-sending issue: WM_DDE_EXECUTE goes out but file is NOT opened in the IDE.

The window message version caused more havoc than it could solve. While collecting DDE targets worked more reliably, it interfered with sending commands.

The finding no instances symptom was caused by the list connect, not Delphi's DDE server; adding error handling revealed that:

      convlist := DdeConnectList(ddeid, svchandle, topichandle, 0, nil);
      If convlist = 0 Then
        Raise EDelphiVersionException.Create('Retrieving the list of Delphi DDE servers failed, DDE error ' + DdeGetLastError(ddeid).ToString);

threw DMLERR_NO_CONV_ESTABLISHED when something was "locked up".

I'm now playing around with

  res := DdeInitializeW(ddeid, nil, APPCMD_CLIENTONLY, 0);

Seems to be stable so far.

7 minutes ago, Attila Kovacs said:

ok, here a native approach, finding the dde window with winapi and sending dde messages, no dde@user32 involved

 

let's see, if it's more stable

This idea won't work as intended, as you are sending all the messages to all TPUtilWindows of all instances you'll find. At least for my purpose, where I am separating Delphi IDE instances by versions.

The theory is interesting though. You think one Delphi instance can have more DDE servers, which are constantly being destroyed / created?

7 hours ago, Attila Kovacs said:

also, I dropped vcl.ddeman and initialize DDE myself

And with this, you made all calls work fine from a background thread!

1 hour ago, Attila Kovacs said:

something is still not ok, after 4 calls the IDE won't respond anymore

I'm using the code to start an IDE if nothing was detected and open a file in it. Once the first instance can not even be detected anymore I still can control the second just fine. This makes me believe the issue is with the DDE server of a specific instance.

There are two options.

Either we lock up Delphi's DDE server with our code, or Delphi's DDE server is buggy and crashes by itself.

Once an instance locks up I'll see if I can double-click on a file in Explorer to open it.

Btw, stability issue seem to be solved by calling DdeDisconnectList when finished. I'll push an update soon.

21 hours ago, Attila Kovacs said:

Here.

 

This queries all the running bde's and opens a file in every one.

This is a good start, you have to find out which is which version somehow, I have no more time for that.

DDE-test.7z

I took the liberty to include everything in a small, easy to use package: https://github.com/aehimself/AEFramework/blob/master/AE.DelphiVersions.pas

Usage:

Var
  dv: TDelphiVersions;
  v: TDelphiVersion;
begin
  dv := TDelphiVersions.Create;
  Try
    For v In dv.InstalledVersions Do
      If v.IsRunning Then
        v.OpenFile('C:\a.pas');
  Finally
    FreeAndNil(dv);
  End;
End;

Multiple DDE queries clearly lock something up, I suspect calling .IsRunning in an infinite loop will turn from True to False after a while.

Also, OpenFile will raise an AV if there are no instances. I'll add some error handling later.

Had that in mind but I'm afraid of using version numbers, as you have to consider patch levels in each version. I guess it might change bds.exe version number too.

3 minutes ago, Attila Kovacs said:

here is the executable name

 

 

 

  Buffer: array [0 .. 4095] of Char;
  Modules: array [0 .. 255] of THandle;
  PID: DWORD;
  exePathLen: integer;
  hProcess: THandle;
  nResult: boolean;
  nTemp: Cardinal;
  aService, aTopic: WORD;

....    

if GetWindowThreadProcessId(ci.hwndPartner, PID) <> 0 then
    begin
      hProcess := OpenProcess(PROCESS_QUERY_INFORMATION or PROCESS_VM_READ, False, PID);
      if hProcess <> 0 then
      begin
        nResult := EnumProcessModules(hProcess, @Modules[0], Length(Modules), nTemp);
        if nResult then
        begin
          nTemp := GetModuleFileNameEx(hProcess, 0, Buffer, SizeOf(Buffer));
          Memo1.Lines.Add(Buffer);
        end;
      end;
    end;

...

 

Yep, similar to my approach. I moved this to a separate function though, but logic is the same.