Revisiting TThreadedQueue and TMonitor

[Anders Melander 1783]

On 1/29/2021 at 11:19 AM, dkounal said:

it is the line: "TMonitor.Enter(FQueueLock);" It is not random. It happens all the time the first time it is used.

Can you think of a reason? 

Since FQueueLock is instantiated in the TThreadedQueue constructor and destroyed in the destructor my guess would be that you are calling PushItem on a TThreadedQueue instance that has been destroyed - or not yet instantiatred.

If you can reproduce in the debugger, try placing a breakpoint on the place where you create the TThreadedQueue and one on the place where you destroy it. This way you should be able to verify that things are called in the order you expect.

[dkounal 3]

47 minutes ago, Anders Melander said:

Since FQueueLock is instantiated in the TThreadedQueue constructor and destroyed in the destructor my guess would be that you are calling PushItem on a TThreadedQueue instance that has been destroyed - or not yet instantiatred.

If you can reproduce in the debugger, try placing a breakpoint on the place where you create the TThreadedQueue and one on the place where you destroy it. This way you should be able to verify that things are called in the order you expect.

I had the same though, but:

1. I can not reproduced it in any of my computers, nor inside the development IDE.

2. It happens to a small number of computers of clients and in a couple of them it is reprodusable.

3. I got this line of error from Eurekalog and I am not it is the correct line of the error

I am testing now the cocinasync Tqueue<T>

[Dalija Prasnikar 1396]

2 minutes ago, dkounal said:

I had the same though, but:

1. I can not reproduced it in any of my computers, nor inside the development IDE.

2. It happens to a small number of computers of clients and in a couple of them it is reprodusable.

3. I got this line of error from Eurekalog and I am not it is the correct line of the error

I am testing now the cocinasync Tqueue<T>

Random errors like that suggest that some of your code is not thread safe and originating point of your problems may not be in the place where it finally blows up, but it gives you some hints to inspect your code that uses ThreadedQueue in broader context and possible code paths before you land on the error.

 

Using different queue will not magically fix your problem, because problem is not in TThreadedQueue but in your code. You might get different behavior, and it might crash more often or not, but it will not be fixed until you find and fix real issue.

[Anders Melander 1783]

27 minutes ago, dkounal said:

I got this line of error from Eurekalog and I am not it is the correct line of the error

MadExcept has an option to compile with memory overrun/underrun checks enabled. It makes heap allocation really slow and add a lot of overhead but it's good if you suspect a memory overwrite and have no clue about where the problem originates.

I don't know if Eurekalog has something similar.

Edited February 8, 2021 by Anders Melander

[dkounal 3]

2 hours ago, Dalija Prasnikar said:

Random errors like that suggest that some of your code is not thread safe and originating point of your problems may not be in the place where it finally blows up, but it gives you some hints to inspect your code that uses ThreadedQueue in broader context and possible code paths before you land on the error.

 

Using different queue will not magically fix your problem, because problem is not in TThreadedQueue but in your code. You might get different behavior, and it might crash more often or not, but it will not be fixed until you find and fix real issue.

I fully agree and I looking into it. It is an old project that slowly undergoes changes to be able to be used with the FMX framework too.

The problem started when I replaced the messaging part between a thread and the UI that in the past TOmniMessageQueue where used. I can confirm that no access exists for ThreadedQueue that can cause problems.

For the sure, the project still uses other ThreadedQueues and Tcriticalsections with other threads and I should check them all. I will check again and I will report. For the moment I have not an error report with cocinasync  but it is too early to be sure.

[dkounal 3]

On 1/29/2021 at 12:19 PM, dkounal said:

Using TThreadedQueue<T> in Delphi 10.4.1, I have a couple of computers with windows 10 running an application of mine that have an Access violation error in line 7917 of system.generics.collections as reported by Eurekalog.

it is the line: "TMonitor.Enter(FQueueLock);" It is not random. It happens all the time the first time it is used.

Can you think of a reason?

function TThreadedQueue<T>.PushItem(const AItem: T; var AQueueSize: Integer): TWaitResult;
begin
  TMonitor.Enter(FQueueLock);
  try
    Result := wrSignaled;
    while (Result = wrSignaled) and (FQueueSize = Length(FQueue)) and not FShutDown do
      if not TMonitor.Wait(FQueueNotFull, FQueueLock, FPushTimeout) then
        Result := wrTimeout;

    if FShutDown or (Result <> wrSignaled) then
      Exit;

    FQueue[(FQueueOffset + FQueueSize) mod Length(FQueue)] := AItem;
    Inc(FQueueSize);
    Inc(FTotalItemsPushed);

  finally
    AQueueSize := FQueueSize;
    TMonitor.Exit(FQueueLock);
  end;

  TMonitor.Pulse(FQueueNotEmpty);
end;

 

Just to comment that the problem was resolved after removing a Tevent that was used by a function.... And the renovation continues with excellent results and Delphi 10.4.2 now.

I also returned back to TThreadedQueue<T>

The books from Dalija Prasnikar (Delphi Event-based and Asynchronous Programming) and Primož Gabrijelčič (Delphi High Performance) helped a lot.

Thank you all for the help

Edited March 3, 2021 by dkounal