Gerben Abbink

I want to count newlines in a huge string (C-like string that ends with a #0), like this: while P^ <> #0 do begin case P^ of #10: begin NewlineCount := NewlineCount + 1; P := P + 1; end; else begin P := P + 1; end; end; end; The string is huge (5 GBytes) so I want to do thing in parallel. I tried Delphi's TParallel.For() but was really slow. OmniThread's Parallel.For() does not support Int64 (and is probably also slow?). So I came up with this: // Create 4 tasks (my CPU has 4 cores) Task1 := CreateTask(TaskProcedure); Task2 := CreateTask(TaskProcedure); Task3 := CreateTask(TaskProcedure); Task4 := CreateTask(TaskProcedure); // Give each task 1/4 of the string Task1.SetParameter('P', P + 0 * Count / 4); Task1.SetParameter('Count', Count / 4); Task2.SetParameter('P', P + 1 * Count / 4); Task2.SetParameter('Count', Count / 4); Task3.SetParameter('P', P + 2 * Count / 4); Task3.SetParameter('Count', Count / 4); Task4.SetParameter('P', P + 3 * Count / 4); Task4.SetParameter('Count', Count / 4); // Return values Task1.SetParameter('PNewlineCount', @NewlineCount1); Task2.SetParameter('PNewlineCount', @NewlineCount2); Task3.SetParameter('PNewlineCount', @NewlineCount3); Task4.SetParameter('PNewlineCount', @NewlineCount4); // Run the tasks in parallel Task1.Run(); Task2.Run(); Task3.Run(); Task4.Run(); // Wait for each task to end Task1.WaitFor(INFINITE); Task2.WaitFor(INFINITE); Task3.WaitFor(INFINITE); Task4.WaitFor(INFINITE); // Total NewlineCount := NewlineCount1 + NewlineCount2 + NewlineCount3 + NewlineCount4; This is the actual thread: procedure TaskProcedure(const OmniTask: IOmniTask); var Count: Int64; NewlineCount: Int64; P: PChar; PEnd: PChar; begin P := OmniTask.Param['P'].AsPointer; Count := OmniTask.Param['Count'].AsInt64; PEnd := P + Count; NewLineCount := 0; while P <> PEnd do begin case P^ of #10: begin NewlineCount := NewlineCount + 1; P := P + 1; end; else begin P := P + 1; end; end; end; (OmniTask.Param['PNewlineCount].AsPointer)^ := NewlineCount; end; This works really great! It is indeed almost 4x faster (actually 3x) than the original non-Parallel procedure. Actually, if I create 8 tasks instead of 4 it is even faster, even if my CPU only has 4 cores. I have some questions: I create 4 tasks for a CPU with 4 cores, but I am not sure if each task is actually a different thread. Should I create 4 threads instead of 4 tasks? Is that possible in OmniThread? The main thread does nothing and waits for each task to end (Task.WaitFor(INFINITE)). Does this mean that 1 CPU core is doing nothing? Should I calculate part of the problem in the main thread?

You are right, this is wrong: NewlineCount1 = Count / 4; NewlineCount2 = Count / 4; NewlineCount3 = Count / 4; NewlineCount4 = Count / 4; Now I do this: NewlineCount1 = Count div 4; Count := Count - NewlineCount1; NewlineCount2 = Count div 3; Count := Count - NewlineCount2; NewlineCount3 = Count div 2; Count := Count - NewlineCount3; NewlineCount4 = Count div 1; Count := Count - NewlineCount4; I will also look at Stackoverflow.