I thought your code that calls Application.ProcessMessages runs in the main thread. (Otherwise you really should not do that.)

If it is running in a background thread, I don't know. I would have to see a working (and as minimal as possible) test program that exhibits your problem to tell you more.

Re 1: I assumed FTask is your task controller. So reading from its Comm property would read messages sent FROM the task. 

Re 2: Task.Comm.Receive runs in the task thread and it DOES receive messages that were sent TO the task.

When you send a message from a background thread to the main thread, the message is inserted into internal message queue and a Windows message is dispatched to a hidden window created by the main thread. (Actually, by the thread that created the task, but in your case this is the main thread.) When this hidden window receives and processes the message, all messages from the internal message queue are fetched and processed.

The Application.ProcessMessage call triggers the message loop so Windows messages may be processed.

Similarly, if a background thread creates OTL task, that background thread (the owner) must call MsgWaitForMultipleObjects[Ex] so that messages are processed. (Alternatively, if that background thread (the owner) is itself an OTL thread, you can create it with modifier .MsgWait.)

IOW, if you want to use messaging subsystem, messages must be processed. 

You can pump the FTask.Comm channel manually during the wait. Instead of Sleep(10) you can call FTask.Comm.Receive(msg, 10). This will return False on no message and True if there was a message in the queue (in which case it is fetched and stored in the `msg` parameter.) Then you will not need Application.ProcessMessages.

As expected. I think we now all agree on how it works 🙂

2 minutes ago, pyscripter said:

...is not explicitly assigned to any variable so its scope is the function scope.

 

Indeed. That's what I was trying to say four posts back 🙂

@Dalija Prasnikar 

Test started
TTestObj.Create
obj = TTestObj
End of nested scope
Test completed
TTestObj.Destroy

Makes sense.

In your code an 'inline var' is an interface of type `IShared<TTestObj>` and is destroyed at the end of the owning scope (begin..end).

In my code, the 'inline var' is of type `TTestObj`. The `IShared<TTestObj>` interface is just something that was created during the execution and is owned by the method itself.

3 minutes ago, Dalija Prasnikar said:

You are wrong... object is destroyed at the end of the inner scope... but your output  'End of nested scope'  is still executed within inner scope.

Nope. Interface is cleared - and object destroyed - in the final `end` of the method. I verified with the debugger.

I ran a test in Rio and it works quite well:

type
  TTestObj = class
  public
    constructor Create;
    destructor Destroy; override;
  end;

constructor TTestObj.Create;
begin
  inherited Create;
  Writeln('TTestObj.Create');
end;

destructor TTestObj.Destroy;
begin
  Writeln('TTestObj.Destroy');
  inherited;
end;

procedure Test;
begin
  Writeln('Test started');
  begin var obj := Shared.Make(TTestObj.Create)();
    Writeln('obj = ', obj.ClassName);
    Writeln('End of nested scope');
  end;
  Writeln('Test completed');
end;

Output:

Test started
TTestObj.Create
obj = TTestObj
End of nested scope
Test completed
TTestObj.Destroy

Inline var object is not destroyed at the end of scope, but at the end of the method. Still, that's something I more or less expected. Other than that, the code works fine.

For the record, this can also be achieved with the "traditional" syntax. Delphi will kindly keep the interface alive until the end of the method:

procedure Test;
var
  obj: TTestObj;
begin
  Writeln('Test started');
  obj := Shared.Make(TTestObj.Create)();
  Writeln('obj = ', obj.ClassName);
  Writeln('Test completed');
end;

And there's also a bit slower but shorter variation to create a shared object:

Shared<TTestObj>.Make()

Just saying 😉

@pyscripter You don't need a temp variable at all:

procedure MeasureSpring;
var
  i: Integer;
begin
  with Shared.Make(TStringList.Create)() do
    for i := 1 to ADD_COUNT do
      Add(i.ToString);
end;

Just ... with ... brrrr, terrible.

This may also work (didn't test):

procedure MeasureSpring;
begin
  // something ...
  begin
    var s := Shared.Make(TStringList.Create)();
    for var i := 1 to ADD_COUNT do
      s.Add(i.ToString);
  end;
  // something ...
end;

I would really like Embarcadero to enhance `with` block in that direction, so I could write:

procedure MeasureSpring;
begin
  with var s := Shared.Make(TStringList.Create)() do
    for var i := 1 to ADD_COUNT do
      s.Add(i.ToString);
end;

This should compile to the same code as my second example. (Except that at the moment it is not support.)

And sometimes fighting with bugs becomes too much and you just remove some functionality for specific subset of compilers. That's why OTL uses OTL_Generics and OTL_GoodGenerics.

In my opinion, the best solution is to use jedi.inc and then IFDEF against features (SUPPORTS_OVERLOAD, SUPPORTS_GENERICS, HAS_UNIT_VCL_THEMES ...) and not against Delphi versions.

Now also available on GetIt.

Why don't you test and tell us? 🙂

(I have no idea but it would be good to know.)

And just to prevent misunderstanding - my longer solution was not attempting to speed up the sorting but the whole process.

Indeed, TStopwatch.ElapsedMilliseconds works great.

Loading 2800 files in 4 seconds doesn't seem slow to me ...

Sorting is damn slow, though. Maybe IntroSort from Spring4D can do it better. I know @Stefan Glienke introduced some optimizations for fast element swapping (which should help with strings).

How much time do you want to spend with this?

The fastest solution would definitely be:

• Load each file into one large block of memory.
• Scan the file and find all strings. Put the offset to each string into a list.
  • Replace each #13#10 with #0#0 during the process.
• Sort this list with a custom comparison method which follows the offset into the large block of memory and uses that string for comparison.
  • You can treat strings as PChars as they will be #0 terminated.
• Note: allocate one byte more than the file size for each block and put #0 in the last byte to ensure that the last string is properly terminated.

After you have that working, you can parallelize the loading and scanning process for additional fun.

"Did you measure it?" 🙂

I can confirm that sorting at the end is faster. Before we start guessing, however, you should do some profiling.

Version 3.07.6 brings official support for Delphi 10.3 Rio, few changes and bugfixes.

You can get it now on git, download the ZIP archive, install it with Delphinus or with GetIt (soon).

For more information, visit OmniThreadLibrary home page or write your question on this forum.

New features

• Implemented TOmniValue.LogValue.
• Implemented TOmniBlockingCollection.FromArray<T>, .FromRange<T>, and .AddRange<T>.
• Added timeout parameter to TOmniMREW.TryEnterReadLock and TOmniMREW.TryExitReadLock.

Bug fixes

• Fixed race condition in TOmniBaseBoundedQueue.RemoveLink which could cause TOmniBaseBoundedQueue.Enqueue to return False when the queue was empty.
• Fixed race condition between TOmniResourceCount.[Try]Allocate and TOmniResourceCount.Release. [tnx to Stephen Melnyk]
• ThreadData is destroyed in the worker thread and not in the thread pool management thread.
• Reduced hints&warnings.

Just a quick note - last OTL release compiles and works fine in Rio.

here will be a new release with some bugfixes and official Rio support in few days.

30 minutes ago, Kryvich said:

If in your (or my) example the step size would be 3, we would get an infinite loop.

As I have said:

Quote

 Such code ... will eventually cause you big problems.

Of course, Andre. There are many ways of doing this in a clean manner. You can also write a nice iterator and call

for i in Range(1, 10, 2) do

That was not a point of the post, though. I wanted to point out one shady part of our bellowed language.

Today I was porting some legacy code and noticed a weird warning:

Weird warning, I thought. Obviously the loop variable can be passed as a var parameter as the code compiles. Why a warning and not an error, then?

Stefan later reminded me that warnings are quite extensively covered in the documentation. Indeed, the docs for W1015 clearly state:

This is a warning because the procedure which receives the control variable is able to modify it, thereby changing the semantics of the FOR-loop which issued the call.

Ah! So they actually want to say “… FOR-Loop variable ‘i’ should not be …”

Furthermore, this brings several interesting possibilities to the table. For example, we can finally write for..to..step loops in Delphi!

program ForInStep;

{$APPTYPE CONSOLE}

{$R *.res}

uses
  System.SysUtils;

procedure Step(var loop: integer; step: integer);
begin
  loop := loop + step - 1;
end;

var
  i: integer;

begin
  for i := 1 to 10 do begin
    Writeln(i);
    Step(i, 2);
  end;
  Readln;
end.

Indeed, this works!

Danger, Will Robinson!

To be clear, I do not recommend writing such code. If you get warning W1015, you should rewrite the code. Such code is hard to maintain and will eventually cause you big problems.

I certainly did fix the original code although the method that was called from the for loop did not modify its var parameter. (It was declared var and not const because the method was written before const parameters were introduced to Delphi. Oh, the joys of legacy code!)

Looks good to me.

Stays fine if I resize the form.

If you want me to test something specific, let me know.

Primož

Can do.

More data, some old, some new.

Firstly, two very old CPUs (the oldest I could find in the company):

This pattern repeats very consistently every 64 bytes (size of cache line):

Very interesting pattern but the worst thing is the terrible slowdown when memory access crosses the cache line.

Similar data can be seen in a Xeon of a similar age:

For a moment I thought I used the wrong data files - that's how similar both results are! 

And now a suprise! A very modern & fast AMD Ryzen Threadripper:

Wow! The cache line is only 32 bytes and memory access across that line is still slow! Interestingly, accessing 4-aligned 8-byte data in 64-bits works great even when straddling cache line.

MemAtomic proves that cache line is only 32-byte:

1: 
2: 15 31 47 63 79 95 111 127 
4: 13 14 15 29 30 31 45 46 47 61 62 63 77 78 79 93 94 95 109 110 111 125 126 127 
8: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 

No wonder Intel is still a king for non-optimized software!