Primož Gabrijelčič

Thank you, @Dalija Prasnikar, I just wanted to post exactly that 🙂

New release should go out this weekend. 

Indeed, OnTerminated is the correct approach.

Does this not already work out of the box? If a task is finished, WaitForAll should now that.

Call stack depth is configurable in FastMM4.

{------------- FullDebugMode/LogLockContention constants---------------}
const
  {The stack trace depth. (Must be an *uneven* number to ensure that the
   Align16Bytes option works in FullDebugMode.)}
  StackTraceDepth = 19;

Check the stack trace to see where this critical section is allocated from.

If this is not possible with the FastMM that comes with Delphi (frankly, I don't use it and I have no idea), use FastMM from git and define FullDebugMode conditional.

This is not possible at the moment. You have to implement your own registration mechanism - register a task in some structure when you create it, deregister it when it is completed.

Oh, that was you! Big thanks!

For the record - I'm trying to keep the book in sync with the OTL most of the time.

5 minutes ago, Attila Kovacs said:

For example, some of the exceptions are arriving as "string" into the final output collection instead as "exception".

Without having a reproducible example, I have no idea.

5 minutes ago, Attila Kovacs said:

Do I need a new pipeline?

Yes, you need a new pipeline. After a pipeline goes into CompleteAdding state, it cannot be "revived".

3 hours ago, Primož Gabrijelčič said:

 Alternatively, I may move this functionality into DSiWin32.VCL ...

Done. OmniThreadLibrary now uses only the DSiWin32 part and ignores DSiWin32.VCL.

Ah, I get it.

I don't like to break compatibility but this is something that could be useful, indeed. Alternatively, I may move this functionality into DSiWin32.VCL (which may be altogether a better idea).

Defining NoVCL where? I don't understand.

Quote

It's currently tied to the vcl (only in DSiWin32.pas)

I have fixed that (just now). DSiWin32 no longer uses Vcl.Graphics if symbol NoVCL is defined.

Oh, a classic mistake 🙂 Been there, done that.

Sure, TOmniBlockingCollection is fine.

I don't know. Show us the code. You are probably doing something after that "WaitFor" and I don't know what.

Quote

where is no waitfor() but processing output right after "CompleteAdding", which is also confusing.

What is confusing here? You don't have to wait for pipeline to terminate (WaitFor) to start processing its output.

24 minutes ago, Lars Fosdal said:

@Primož Gabrijelčič From the wishful thinking department:
I'd love to see OTL evolve to support Linux/MacOS, iOS and Android.
Hence, I'd love to see solutions that do NOT use Windows messaging - or at least hide it inside the notification implementation.

I would love that too!

You can probably use PostThreadMessage Windows API to post a message directly to the main thread. Then you process this message in your process loop.

Of course you can run a pipeline without a WaitFor.

You have different options to detect when a pipeline has finished its work.

a) The main program can count the number of items sent to the pipeline and number of items returned from. (If there is a simple correspondence between two - for example if each input produces exactly one output.)

b) The pipeline itself can detect that it has no more work and then it can signal this to the main program.

When you detect a terminating condition, you can shut down the pipeline (with WaitFor) and you'll done.

See the "folder scanner" in OTL examples folder for an example of the b) technique or read this chapter of the book: http://www.omnithreadlibrary.com/book/chap10.html#howto-webDownload

There is none. Only the main thread should update the UI in a VCL application so if it is blocked, you're out of options.

In theory, you could create a window purely by the Windows API and use it to show the progress, but that is probably much to much work. 

Better solution would be to not block the VCL application at all.

Presentations and code are available here. I'll send you a link to the video recording as a private message.

When the WorkItem is freed. Usually that happens after it is dispatched to your notification handler (OnRequestDone, OnRequestDone_Asy).

That helps, thanks!

You are missing a reproducible test case 😉

Sincerely, I have no idea. If the example works for you (you tested it, probably?) and the application doesn't, you'll have to find what that difference is.

I'm usually doing such logging in a manner similar to the one Kas Ob. presented.

I establish a logging queue. Usually a TOmniQueue as it does not require locking. 

Then I create a logging thread. In essence it does just:

while logQueue.Get(msg) do
  Log(ms);

Each part of the program that wants to log, just creates a `msg` record containing whatever info you need to perform the logging (message, severity, timestamp, target file ...) and pushes it into the logging queue. That's it.

On 2/10/2020 at 9:07 PM, Tommi Prami said:

Fire up tasks in parallel and feed work to them, way or other, pass work log back and so on. 

My book Hands-On Design Patterns with Delphi covers a Pipeline pattern which I fully implemented in PPL. It may give you some ideas on how to approach your problem.

The source code is freely available on GitHub.