Search the Community

I have an application which continuously receives websocket data and proccess them in the background on worker threads. Some data is critical and must be proccessed as quickly as possible. Once I receive the websocket data I fill a TObject descendent with the information and push it to the corresponding worker thread, which proccess and frees the object. I use Delphi 12. I would like to reduce as much overhead as possible in the flow, and beside the business rules, I believe there is an overhead in my worker thread consumer implementation, especially because I work with TObject descendants to transport the data. Also there has TObject cloning when the communication occurs from worker thread to worker thread, because each worker thread owns the queue objects lifetime, so I need to send a copy for each worker thread. I decided to make a benchmark to check by myself the differente in the differet approachs that I know about threading queues and know what has the best performance, attached has the benchmark I build. Nowdays I use CustomQueueObject.pas queue model in prod. In the example, has 4 examples of threading consumer queues: -A thread with TQueue<TObject>, TSempahore and TCritical Section; -A thread with TQueue<Pointer>, TSempahore and TCritical Section; -A thread with internal TThread queue processing TObject descendents, without events and sync objects; -A thread with internal TThread queue processing Pointers, without events and sync objects; After making tons of tests, for my surprise the fastest consumer queue is the example in CustomQueueObject.pas (which I already use), even with more creation/deletion and access control (sync object)... what brought me here to ask more experienced developers if I doing something wrong (according my examples as is my base). Mainly on Pointer examples as I rarely use pointers but I willing to change if its better. In my mind the internal thread queue would much more efficient for the reasons said above... I also thought that work with Pointers could highly improve the overal performance as in the workflow since I would have a single reference and only freed in the final. Could I be measuring the performance incorrectly? Thanks in advance! MultithreadingQueueBenchmark.zip

I new using TClientDataSet and would like some help. My application fetch heavy stream data from an external web-socket like API (from multiple channels), proccess it asap in worker threads and then show to the user when requested... nowdays I do it manually with lists, arrays and iterate to populate the components (manually). I want to improve and standartize the data in the application memory, the approach I thought is to go with TClientDataSet. The problem is that the application works with a heavy multithread system (which works pretty well, its not the problem), and I know that the TClientDataSet is not thread-safe, and could have alot of problems to deal with, so I would like to ask who already faced this problem before, if I on the right way. My approach: • The worker threads (which will receive the data to proccess) will write in a own TClientDataSet, only the worker thread may write data in its dataset; • Operations (insert/update/delete) in worker thread TClientDataSet wouldn't be synchronized; • Operations and cloning will be protected by a sync object (TCriticalSection like); • The UI that must shown data may clone the worker thread TClientDataSet to have a read-only copy, which can be linked in VCL controls (like TDBGrid) and also iterated, filtered, etc. Anyone that experienced this scenario (or similar) can say if I on the right way to avoid thread synchronization and deadlock problems dealing with TClientDataSet in a multi-thread system?

Hi everybody, I'm using Delphi 12 on Windows for Desktop applications. I've an FMX project done in single threaded mode (classical without threads) with timers and I have a problem with animation of objects inside frames! The project is structured like that : A main form with 2 timers and 3 layouts (Top, Main, Bottom). 1 - The first timer is in charge to display successively TFrames regarding a process inside the layouts 2 - The second one is in charge of the standby mode management A second form with 5 timers in charge of the communication with a PLC 1 - The 5 timers have different frequencies in order to read different values of the PLC (500 ms, 1s, 1.5s, etc) In the main form, on specifics frames, I've some objects (like TArc or TRectangle, etc.) that are animated by using TFloatAnimation. My problem is that these animations are lagging. Trying to analyse the problem I made a try without timers and it seems to work smoothly but it is hard to conclude if it is only the timers! I would like to create some threads to eliminate the timers and my idea is to create one thread by frequency of reading. But when I will call the method to read from the "Execute" function of the thread, the method called will be located inside the second form in a specific class (but the method is not located in a visual component of the FMX library). So my questions are: Should I use "Synchronize" even if the method called is not a visual component ? I'm afraid of the fact that it could slow down the application as before by the timers if each thread is all the time Synchronizing to make the different read methods calls! To replace the 2 timers of the main thread, I imagined the use of 2 more threads to send events to the main form to let it know when it is needed to change of frame or going in standby mode. Is there a good idea? Is there really the good approach? I'm a little bit lost never stopping to think to do it in the better way. Many thanks to all in advance for your contributions.

i have this code here: fTask: ITask; fTask := TTask.Create( procedure begin while not True do Application.ProcessMessages; TThread.Synchronize(TThread.Current, procedure begin // update ui end); end); fTask.ExecuteWork; // this will stop here.. until [fTask] finish work !! In android didn't wait untill Task Complete !!

Hello Community, since last week I have been trying to let a TsslWSocket and a TsslWSocketServer communicate with each other. I want to get a long-term stable connection between a TCP client and a TCP server that is secured with TLS1.3. However, after a runtime of a few minutes, the data transfer breaks down. There is a client thread and a server thread. In the respective Execute method I create the socket and initialize it, call Listen (server) or Connect (client) and run the MessageLoop. After a successful SSL handshake, my client sends 128KB data packets to the server. The data arrives at the server, TwSocketClient.onDataAvailable is triggered, the bytes can be read there with TwSocketClient (Sender).Receive (pBuffer ...). When the server has received 128KB, it sends an "ACK" packet to the client (An AnsiString "ACK", 3 bytes). When the client receives the packet, it sends again 128KB of testdata. This works a few thousand, sometimes some tenthousands times, until TwSocketClient.onDataAvailable is suddenly no longer called. There is no OnSessionClosed, no onError, no onException and also no onBgException that would indicate a problem. Apparently the problem is on the client side: A running Wireshark shows an [ACK] packet from the server to the client as last entry. Further clicks on the "Manual Send" button do not lead to any further lines in the Wireshark log. When this state has been reached another client can connect to the server without any problems, so the server is not completely dead. I have no idea what I could do wrong ... Maybe it's not ok to send in the context of another thread? But how is that supposed to work when the client thread is blocked by the MessageLoop? Simultaneous web browsing leads to the problem faster. When I look at any website, it usually takes less than half a minute to see the problem. My test project is attached. I would be interested to know if you have observed the same problem and if anyone has a guess as to what might be the cause ... The project is complete, the Win32 folder also contains the certificate used for testing ... Usage: Enter your own IP address, press the "btCreateStart" button, then the "btSendData Manual Send" button. The "logICStest.log" file in the program directory grows until the problem occurs ... Thanks in advance... P.S: I have already tried not to send after receiving the ACK packet, but cyclically via timer or thread. If you can select both in the test project with checkboxes, the result is the same. The problem does not only occur with TLS1.3, also with TLS1.2. I even think it shows up without encryption, just takes longer ... I crossposted this in the german Delphi-Praxis in german language (https://www.delphipraxis.net/205223-stabile-tcp-verbindung-mit-ics-ich-kriegs-nicht-hin.html)... Test Overbyte ISC Client und Server.7z

Using Delphi 10.3.2 I've encountered the following problem (specifically on Windows, not sure about other platforms): After using TParallel or TTask, not all threads in the thread pool return to idle as expected. I've prepared the following simple example: procedure TForm.Button1Click(Sender: TObject); var SW:TStopWatch; procedure Load; begin TParallel.For(0, 99999999, procedure(i: Integer) var T:Single; begin T:=Sin(i/PI); end); end; begin Memo1.Lines.Add('PPL Test ---------------'); Memo1.Lines.Add('Before: '+TThreadPoolStats.Current.Formatted); SW:=TStopWatch.StartNew; Load; Memo1.Lines.Add('Finished in '+SW.Elapsed.ToString); Memo1.Lines.Add('After: '+TThreadPoolStats.Current.Formatted); Memo1.Lines.Add('------------------------'); end; Repeatedly pressing the button results in the following output: PPL Test --------------- Before: 0 Worker Threads (56 .. 1400) / 0 Idle / 0 Retired / 0 Suspended / Avg. CPU = 0 / Curr. CPU = 0 Finished in 00:00:09.8423686 After: 56 Worker Threads (56 .. 1400) / 43 Idle / 0 Retired / 0 Suspended / Avg. CPU = 30 / Curr. CPU = 100 ------------------------ PPL Test --------------- Before: 56 Worker Threads (56 .. 1400) / 43 Idle / 0 Retired / 0 Suspended / Avg. CPU = 36 / Curr. CPU = 0 Finished in 00:00:04.7841342 After: 65 Worker Threads (56 .. 1400) / 50 Idle / 0 Retired / 0 Suspended / Avg. CPU = 43 / Curr. CPU = 50 ------------------------ PPL Test --------------- Before: 65 Worker Threads (56 .. 1400) / 50 Idle / 0 Retired / 0 Suspended / Avg. CPU = 42 / Curr. CPU = 0 Finished in 00:00:04.7090980 After: 74 Worker Threads (56 .. 1400) / 55 Idle / 0 Retired / 0 Suspended / Avg. CPU = 42 / Curr. CPU = 48 ------------------------ PPL Test --------------- Before: 74 Worker Threads (56 .. 1400) / 55 Idle / 0 Retired / 0 Suspended / Avg. CPU = 38 / Curr. CPU = 0 Finished in 00:00:02.3904253 After: 74 Worker Threads (56 .. 1400) / 50 Idle / 0 Retired / 0 Suspended / Avg. CPU = 57 / Curr. CPU = 98 ------------------------ PPL Test --------------- Before: 74 Worker Threads (56 .. 1400) / 50 Idle / 0 Retired / 0 Suspended / Avg. CPU = 51 / Curr. CPU = 0 Finished in 00:00:05.2865123 After: 84 Worker Threads (56 .. 1400) / 55 Idle / 0 Retired / 0 Suspended / Avg. CPU = 44 / Curr. CPU = 41 ------------------------ PPL Test --------------- Before: 84 Worker Threads (56 .. 1400) / 55 Idle / 0 Retired / 0 Suspended / Avg. CPU = 36 / Curr. CPU = 0 Finished in 00:00:06.8032435 After: 96 Worker Threads (56 .. 1400) / 64 Idle / 0 Retired / 0 Suspended / Avg. CPU = 36 / Curr. CPU = 32 ------------------------ PPL Test --------------- Before: 96 Worker Threads (56 .. 1400) / 64 Idle / 0 Retired / 0 Suspended / Avg. CPU = 0 / Curr. CPU = 0 Finished in 00:00:07.7921758 After: 111 Worker Threads (56 .. 1400) / 77 Idle / 0 Retired / 0 Suspended / Avg. CPU = 27 / Curr. CPU = 23 ------------------------ PPL Test --------------- Before: 111 Worker Threads (56 .. 1400) / 77 Idle / 0 Retired / 0 Suspended / Avg. CPU = 20 / Curr. CPU = 0 Finished in 00:00:02.3559000 After: 111 Worker Threads (56 .. 1400) / 77 Idle / 0 Retired / 0 Suspended / Avg. CPU = 48 / Curr. CPU = 100 ------------------------ PPL Test --------------- Before: 111 Worker Threads (56 .. 1400) / 77 Idle / 0 Retired / 0 Suspended / Avg. CPU = 48 / Curr. CPU = 0 Finished in 00:00:06.6491982 After: 122 Worker Threads (56 .. 1400) / 87 Idle / 0 Retired / 0 Suspended / Avg. CPU = 33 / Curr. CPU = 100 ------------------------ Even after the first run, only 43 threads in the pool seem to have returned to idle. Instead, with almost every further run, the total number of worker threads is increased to make up the difference. Despite this, the total CPU usage is going down slowly. If I wait for the inactive threads to be automatically retired and run the test again, I get something like this: PPL Test --------------- Before: 0 Worker Threads (56 .. 1400) / -36 Idle / 0 Retired / 0 Suspended / Avg. CPU = 0 / Curr. CPU = 0 Finished in 00:00:09.5530676 After: 56 Worker Threads (56 .. 1400) / 20 Idle / 0 Retired / 0 Suspended / Avg. CPU = 30 / Curr. CPU = 100 ------------------------ A negative number of idle threads? And after the run, only 20 threads are stated as idle again. Further runs result in the behavior described above. If someone could shed some light on all this for me, I would highly appreciate it! Am I doing something wrong here? What do I need to do to get 100% CPU usage on every run, without the thread pool ballooning? Thanks!