aehimself

I only have 10.4.2 so I don't know if this version is available on 10.2, but you can try with

var
 jv: TJSONValue;
 tb: TBytes;
begin
 tb := TEncoding.UTF8.GetBytes(Edit1.Text);

 Try
     jv := TJSONObject.ParseJSONValue(tb, 0, [TJSONObject.TJSONParseOption.IsUTF8, TJSONObject.TJSONParseOption.RaiseExc]);
     jv.Free;
 Except
   On E:EJSONParseException Do
   Begin
     ShowMessage('JSON can not be parsed at path ' + E.Path + ', line ' + E.Line.ToString + ', position ' + E.Position.ToString);
   End
   Else
     Raise;
 End;

Aaaah, okay, it's clear now. I thought LibraryLocation isn't working for you for some reason.

@miab's version is a lot more elegant though, that way all your apps can access their correct version of DB libs without any further effort.

In theory LibraryLocation should work just fine, I'm using it in my main application without issues. There's no reason copying it right next to the executable would be needed.

This might worth looking into.

Can this be reproduced in a minimal app?

Just now, superc said:

When compile Zeos for 64 bit ZComponentDesign not compile for 64 bit with error:

 

[dcc64 Fatal Error] ZComponentDesign.dpk(36): E2202 Required package 'designide' not found. It's normal?

Completely. Pure design time packages can only be compiled in 32 bit as the Delphi IDE itself is 32 bit only.

If the runtime packages already compiled to 64 bit you are good to go. Just install the design time package and start coding 🙂

We have a fairly large C# solution at work, currently using Visual Studio as an IDE. Slow as hell, sluggish and memory-hog, but Intellisense did not stop to function for me; not even once in the past 4 years.

Delphi's LSP was never "stable". I tried D10.2, 10.4, 10.4.2 and now 11.2. Even in relatively small projects (~30-60 units) Code navigation (Ctrl + Click) is uncommon, Code completion usually shows only the default options, wiggly lines everywhere but you still can run your project just fine.

I prefer Delphi and do dislike VS as an IDE, but Intellisense is superior in every way in my opinion. LSP is just still immature.

4 hours ago, RDP1974 said:

you have to setup microsoft visual c runtime before https://learn.microsoft.com/it-it/cpp/windows/latest-supported-vc-redist?view=msvc-170

+1

Also don't forget that some libraries might require LibSSL and LibCrypto to be available.

11 hours ago, superc said:

it seems to be a known bug

 

https://blogs.embarcadero.com/rad-studio-11-2-alexandria-patch-1-available/

The bug you mention has nothing to do with Zeos. Delphi 11 messed up the default library location and attempted to use 32bit DCUs on Win64 platforms. You can even correct it by hand after installation, there was even a post about it here in the D11 thread. I believe this was the faulty entry, just add "\Win64" after it and it starts to work.

10 hours ago, superc said:

I'm tryied now to set LibraryLocation on runtime but I'm obtain same error, it's so frustating. But the question is, on Delphi 11.2 someone use Zeos on 64 bit?

I do, on a daily basis; I'm not connecting to PostgreSQL though (Oracle, MsSQL, MySQL and sometimes SQLite are my main targets). I can confirm that Zeos works on 64 bit with D11.2.

The error above means that the given library was found but can not be used for whatever reason (LoadLibrary failed). I am playing this with MySQL x86 lately, as probably the DLL is so out-of-date (Oracle is not providing a fresh libmysql.dll for a while) that some needed functions do not exist.

If you are absolutely sure the architecture of the DLL matches your project target, try using a more recent version. Or an alternative, if that exists - in my example, libmariadb.dll x86 works just fine.

Edit: consider using Zeos 8. It's not yet considered as "release-ready" officially but it's rock solid and brings some great improvements. Afaik, it mostly lacks documentation only.

Edit-edit: Zeos 8 comes with a D11 package so you can just double-click and install.

As it turns out , manually you can use TJSONValue.ParseValue and TJSONByteReader just like TJSONValue.ParseJSONValue does. But instead of throwing an exception, just mark the wrong items, using TJSONByteReader.OffsetToPos to get the line / character position from the offset.

When an error happens, just skip 1 byte in TJSONByteReader and repeat.

In theory, this could work.

If you use System.JSON and parse with exceptions enabled, it'll tell you where an error is. However, it will only show the first error and not all, but at least it's a built-in option.

1 minute ago, Stano said:

Kinda OT

As a lay amateur, I am puzzled by the fact that there are several files in one BLOB. Too much.
I would save each file separately from the very beginning. I was already taught that with modern DBs the number of records in the table is not a problem.
But I don't know your program, nor its philosophy and function.

There can be multiple reasons

Wrong database design from the beginning

Either the person had no understanding how it should be done or thought that the small amount of data what it will hold won't cause further issues. Then it grew out of control.

To be honest (especially in newer applications) developers tend to completely ignore edge cases, lower hardware specs. "It works on my PC"

Legacy dragging on

This is exactly the case at us at work. Application was created ~20 years ago, when DB backends worked completely different and the architecture was functioning correctly. Now with thousands of tables and TBs of data stored it's not an easy task to start to split these so when a customer complaint comes in, we fix that one. At this speed the redesign won't be done in the next 5 years for sure and expose us to a risk that once done we'll have to start it again due to backend changes.

Can't reproduce

Not just the amount of data but the data itself can cause issues. Generally speaking, due to GDPR and confidentiality we, the developers aren't allowed to access and work with production data. When an export is needed, data is obfuscated before loaded in our test systems and this obfuscation itself is already changing the data. Once I spent days before figuring out that a rendering issue was caused by a BOM left in a BLOB field (customer just uploaded a file instead of using our editor), which was simply trimmed during obfuscation.

As quick as possible, no future in mind

This is especially true for commercial software unfortunately. Customers want more and more, quicker and quicker. In lots of occasions new features are added on a "quantity over quality"-basis. Just get it to work, get us paid, customer is happy - at least for now. This isn't necessarily a management issue as recognizing the need for, as well as optimization itself needs knowledge, time and manpower. Depending on the size, a company can lack many of this.

These are just my 2 cents. Nothing is unfixable, question is always: at what cost.

I implemented the DDE API's execution code and there is a significant difference immediately. If you first open a large project (.dproj) which takes about 5-10 seconds to load, then a form / frame which takes about the same in rapid success, when using WIndows messages the form is often not loaded however the DDE server says it was accepted and processed.

When using the DDE API, both are opened successfully.

Maybe something is missing from the WIndows message version...?

If it doesn't disturb you or you are not opening multiple files right after each other, you can safely ignore this.

@Attila Kovacs If you are interested, I managed to get execution of DDE commands to work with the API so you can ditch all windows message logic:

Procedure TAEDDEManager.ExecuteCommand(Const inCommand: String; Const inConv: HConv; Const inTimeoutInMs: Cardinal = 5000);
Var
  datahandle: HDDEData;
  res: LongInt;
Begin
  datahandle := DdeCreateDataHandle(_ddeid, @PChar(inCommand)[0], Length(inCommand) * SizeOf(Char), 0, 0, CF_TEXT, 0);
  If datahandle = 0 Then
    Raise EAEDDEManagerException.Create('Creating data handle failed, DDE error ' + DdeGetLastError(_ddeid).ToString);

  If DdeClientTransaction(Pointer(datahandle), DWORD(-1), inConv, 0, CF_TEXT, XTYP_EXECUTE, inTimeOutInMs, @res) = 0 Then
    Raise EAEDDEManagerException.Create('Executing command failed, DDE error ' + DdeGetLastError(_ddeid).ToString);

//  If Not DdeFreeDataHandle(hszCmd) Then
//    Raise EDelphiVersionException.Create('Could not free data handle, DDE error ' + DdeGetLastError(_ddeid).ToString);
End;

This way the only method which needs reimporting is DdeInitializeW. inConv can come from DdeQueryNextServer or DdeConnect. As HDATA_APPOWNED is NOT defined as flag, freeing up the data handle will be performed on the DDE server side - that's why that last part is commented out.

As it turns out, the AV isn't caused by CreateProcess or anything related to it. The reason I wasn't able to reproduce it in a small bench application is I have a TDelphiDDEManager class created, which is however unused in the sample, but it calls DdeInitializeW. In the moment I comment that line out (or won't create the DDEManager instance) the AV disappears, PID is shown correctly, everything works like a charm.

Question is, is it because of improper parameterization (like in UnpackDDElParam and FreeDDElParam) or I'm calling that wrong...

Anyway, getting closer 🙂

5 hours ago, Remy Lebeau said:

Using the Addr() intrinsic in that way will give you the memory address of the _TaskDialogIndirect variable itself, not the memory address that the variable holds.  Try inspecting just _TaskDialogIndirect by itself, or at least cast it to a Pointer.

Unfortunately (?) yeah, it has a valid address:

There is also a check around this block in CommCtrl, if the method is not found we wouldn't even come to this block:

    if ComCtl32DLL <> 0 then
    begin
      @_TaskDialogIndirect := GetProcAddress(ComCtl32DLL, 'TaskDialogIndirect');
      if Assigned(_TaskDialogIndirect) then
        Result := _TaskDialogIndirect(pTaskConfig, pnButton, pnRadioButton,
          pfVerificationFlagChecked) // Exception is raised here
    end;

25 minutes ago, Remy Lebeau said:

That means you are trying to call a function through a nil function pointer.

Did you check to see whether _TaskDialogIndirect is nil or not?

I only called ShowMessage, which requires nothing but a string to display. I checked all parameters and confirmed that pszContent indeed contains the PID I'm about to display, but never thought of _TaskDialogIndirect being nil... it's in Delphi VCL after all.

I just checked and yes it seems, it does point to a valid memory address:

Calling UniqueString(_bdspath) before the CreateProcess call still produces an error. Maybe it's even unnecessary...?

https://www.oreilly.com/library/view/delphi-in-a/1565926595/re354.html

"If you cast a string to PChar, Delphi also ensures that the PChar points to a unique instance of the string. If you modify the string through the PChar pointer, the other references to the original string are safe."

I don't know, I am in completely uncharted territory here. Never had to think more of PChar castings than if a method needed it instead of a String.

18 minutes ago, David Heffernan said:

First argument of CreateProcess must be writeable? Is it? 

I was unaware of this, and even MSDN specifies it as IN only. Do you maybe have some references I can look into?

Changing it to

  pc: PChar;
Begin
  [...]

  GetMem(pc, Length(_bdspath) + 1);
  Try
    StrPCopy(pc, _bdspath);

    If Not CreateProcess(pc, nil, nil, nil, False, CREATE_NEW_PROCESS_GROUP, nil, nil, StartInfo, ProcInfo) Then
      RaiseLastOSError;
  Finally
    FreeMem(pc);
  End;

still throws the AV.

I am really baffled by the nature of this issue. Popping up at window messages, and being "execution at address" I'd say it's like a memory corruption. I have doubts though as commenting out the two CloseHandle lines will delay / get rid of these. That clearly points to improper use of CreateProcess...

I'm always using a "subscriber model" when it comes to logging. I have a singleton class, with 2 important methods: Subscribe and Log. You can subscribe with a

TLogProc = Procedure(Const inLineToLog: String) Of Object;

for example, which is only adding the method pointer to a TArray / TList.

Logging uses a critical section and loops through all subscribed loggers, like:

Procedure TMyLogger.Log(const inLineToLog: String);
Var
 logrefproc: TLogProc;
 s: String;
Begin
 TMonitor.Enter(Self);
 Try
  s := '[' + DateTimeToStr(Now)+ '] ' + inLineToLog.Trim;

  For logrefproc In _logrefprocs Do
   Try
    logrefproc(s);
   Except
    // Swallow logger exceptions.
   End;
 Finally
  TMonitor.Exit(Self);
 End;
End;

This way you can call MyLogger.Log from any thread and the message will be distributed to all subscribed loggers. This way, printing to the console window, appending to a TMemo, saving it to a local file and pusing it up to the cloud can all happen - if you wish. As you already seem having some kind of a logger object, moving it to a separate unit and using this logic should suit your needs well. Just make sure the logging methods aren't taking long AND are synchronized, as they will be called by the thread's context! When you have lots of entries I found it easier to add them to a TList<String> object and using a TTimer to dump the contents and empty the list every second or so:

Procedure TLogFrame.InternalLog(const inLogWhat: String);
Begin
 System.TMonitor.Enter(_linestolog);
 Try
  _linestolog.Append(inLogWhat + sLineBreak);
 Finally
  System.TMonitor.Exit(_linestolog);
 End;
End;

Procedure TLogFrame.LogTimerTimer(Sender: TObject);
Begin
 LogTimer.Enabled := False;
 Try
  System.TMonitor.Enter(_linestolog);
  Try
   If _linestolog.Length > 0 Then
   Begin
    LogMemo.Lines.BeginUpdate;
    Try
     LogMemo.Text := LogMemo.Text + _linestolog.ToString;

     CursorToEnd;
     _linestolog.Clear;
    Finally
     LogMemo.Lines.EndUpdate;
    End;
   End;
  Finally
   System.TMonitor.Exit(_linestolog);
  End;
 Finally
  LogTimer.Enabled := True;
 End;
End;

And my OCD forces me to tell you, that

On 12/6/2022 at 2:28 PM, felixeckert said:

 

  if uLogAbstractor.loglvl = LOG_LVL_DBG then
  begin
    if FLogger.FLogDebug then
      memoLog.Lines.Add(FLogger.FLastMsg);
  end else
    memoLog.Lines.Add(FLogger.FLastMsg);

 

can be simplified to:

  if (uLogAbstractor.loglvl <> LOG_LVL_DBG) Or FLogger.FLogDebug then
      memoLog.Lines.Add(FLogger.FLastMsg);

After solving some mysteries in DDE I decided to extend my Delphi instance helper with a new method: to launch a new Delphi instance of a selected version. The theory is simple: launch bds.exe and return the TDelphiInstance object only when the IDE is available (this was removed to further simplify the calls happening).

While the new process starts, I'm getting Access violation at address 00000000 in module 'Project3.exe'. Execution of address 00000000 upon:

- Calling ShowMessage (WinApi.CommCtl : 13107, TaskDialogIndirect)

        Result := _TaskDialogIndirect(pTaskConfig, pnButton, pnRadioButton,
          pfVerificationFlagChecked)

- Application.ProcessMessages loop (Vcl.Forms : 11460, TApplication.ProcessMessage)

            DispatchMessageW(Msg)

- Closing the instance which was created by CreateProcess (error raises in ProcessMessages, just like above)

From the third appearance I believe I'm not using CreateProcess correctly (in theory, after closing all handles my application should have nothing to do with the instance created) but if I move the code out of TBorlandDelphiVersion to the project itself, it just works.

Launching the exe is performed as follows:

Function TBorlandDelphiVersion.NewDelphiInstance: TDelphiInstance;
Var
  StartInfo  : TStartupInfo;
  ProcInfo   : TProcessInformation;
  ec: Cardinal;
Begin
  Result := nil;

  FillChar(StartInfo, SizeOf(TStartupInfo), #0);
  StartInfo.cb := SizeOf(TStartupInfo);
  FillChar(ProcInfo, SizeOf(TProcessInformation), #0);

  If Not CreateProcess(PChar(_bdspath), nil, nil, nil, False, CREATE_NEW_PROCESS_GROUP, nil, nil, StartInfo, ProcInfo) Then
    RaiseLastOSError;

  Try
//    If Not GetExitCodeProcess(procinfo.hProcess, ec) Then
//      RaiseLastOSError;
//
//    WaitForInputIdle(procInfo.hProcess, INFINITE);

    Repeat
      If Not Assigned(Result) Then
      Begin
        Self.RefreshInstances;
        Result := Self.InstanceByPID(procinfo.dwProcessId);
      End;
    Until Assigned(Result);// And ((Result.IDEHWND <> 0) Or Result.FindIdeWindow) And Not Result.IsIDEBusy; // Normally here, removed for simplicity
  Finally
    CloseHandle(ProcInfo.hThread);
    CloseHandle(ProcInfo.hProcess);
  End;
End;

I attached the full source of said class, you can reproduce the issue by...

Uses AE.DelphiVersions;

procedure TForm3.Button1Click(Sender: TObject);
begin
 ShowMessage(ni.ToString);
End;

procedure TForm3.Button2Click(Sender: TObject);
Var
 dv: TDelphiVersions;
 c: Cardinal;
begin
 dv := TDelphiVersions.Create(nil);
 Try
  c := dv.LatestVersion.NewDelphiInstance.PID;
//  Self.Caption := c.ToString; // This works, no exception is raised
  ShowMessage(c.ToString);
 Finally
   FreeAndNil(dv);
 End;
end;

function TForm3.ni: cardinal;
Var
 dv: TDelphiVersions;
begin
 dv := TDelphiVersions.Create(nil);
 Try
  Result := dv.LatestVersion.NewDelphiInstance.PID;
 Finally
//   FreeAndNil(dv); // If you do not free this up, exception is raised in ShowMessage.
 End;
end;

MadExcept reports no buffer under- or overruns, no memory, handle or any other leaks are reported by DeLeaker.
 

I never really used CreateProcess, and never actually managed to produce an execute at address AV up until this point. Can anyone point me to the right direction?

P.s.: Using Delphi 11.2, compiled app is 32 bit, cannot check if the issue will appear anywhere else.

AE.DelphiVersions.zip

Without seeing the code it's really hard to tell what is eating up the memory. I also have zero experience with FireDac, but...

I suspect you have a query, with a DBGrid connected and this query also downloads the BLOB. The issue is that the query will at least download all the blobs for all records which are visible in the grid, worst case it downloads all. This is already using up a huge amount of memory. When you assign the value to a string, you allocate even more memory, resulting in OOM.

What I'd try is to exclude the blob from the main query, add a second query with SELECT blobfield FROM MyTable WHERE ID = x. You can open this manually upon record change, load the data in the RichEdit and close it immediately. That should minimize the memory usage.

https://github.com/aehimself/AEFramework/blob/master/AE.DelphiVersions.pas finally updated, if anyone will need this in the future.

I'll attempt the reporting sometime next week. Never did that before 🙂

@Attila Kovacs

// Need to reimport these two

function UnpackDDElParam(msg: UINT; lParam: LPARAM; puiLo, puiHi: PUINT_PTR): BOOL; stdcall; external user32;
function FreeDDElParam(msg: UINT; lParam: LPARAM): BOOL; stdcall; external user32;

// class procedure TDdeHelper.DdeWndProc
  pLo, pHi: PUINT_PTR; // NOT IntPtr

  GlobalUnlock(pHi^);
  GlobalFree(pHi^);

And your code should be 64-bit compatible 🙂

I'll run some more tests but it seems that now it is finally working.

It's not a threading issue, it's 64-bit issue!

According to MSDN, it's declared as:

BOOL UnpackDDElParam(
  [in]  UINT      msg,
  [in]  LPARAM    lParam,
  [out] PUINT_PTR puiLo,
  [out] PUINT_PTR puiHi
);

In Delphi, however:

Now, we are feeding it an LParam, which is NativeInt.

Time to re-import it, correctly this time 🙂

Your original idea was good, I'm indeed sending WM_DDE_INITIATE first. However attempts to purge the queue is unsuccessful, as there is nothing in the message queue:

  msghwnd := AllocateHWnd(nil);
  Try
    atomservice := GlobalAddAtom(PChar(_service));
    atomtopic := GlobalAddAtom(PChar(_topic));
    Try
      SendMessage(inDDEServerHWND, WM_DDE_INITIATE, msghwnd, Makelong(atomservice, atomtopic));
    Finally
      GlobalDeleteAtom(atomservice);
      GlobalDeleteAtom(atomtopic);
    End;

    While PeekMessage(msg, msghwnd, 0, 0, PM_REMOVE) Do
     Begin
       Sleep(20);
     End;

I split LParam by LoWord and HiWord, in that case FreeDDElParam(msg.message, msg.lParam) throws an AV...

it's like LParam is malformed somehow...