sfrazor

Angus, In a previouse quesiton I asked about high CPU usage. We found that MsgWaitForMultipleObjects worked to resolve the issue. but sleep(0) did not. I tested with single files and transfer speeds were great and CPU was normal. In my mutiple file send scenario, if I remove the MsgWaitForMultipleObjects and let the CPU run at ~50 percent in that main loop, all of the files transfer without issue. No other code changes. I'm really at a loss. Some more info... after every call to any kind of socket IO I immediately call processmessages. SInce this is a console app and will eventually be a DLL the messages need to be addressed. This is something else you pointed out to me some time back. The file send is in its own thread looking for the existence of files, reads them and sends them. Nothing magical. There is a Sleep(5000) inbetween checks.

It works really well for one file at a time. 🙂 Its like its not ready to send when I call the WSSendBinaryStream in a loop but it takes in the data in to a buffer and it sits there. It seems out of sync with the socket state. Its like I need to wait for a state like httpready or something before I call the send. The server is a python server running Flask utilizing websockets. The server has been solid for some time but that doesn't mean there is not an issue. Its like the ICS client isn't receiving an ACK from the last frame sent, if it was infact sent Or the Comms is not in the right state. Or the entire cycle is too fast to process..... Slowing it down with sleeps in between doesn't seem to have any effect..... But I'm not sure how to verify the socket is in the ready-to-send state. I don't hink its fair to ask you to reproduce the client and server. So instead just give this some thought and get back if you come up with something that would be a worth try. I'm still working at it on my end. Again I have to apologize. Everything I do is on an air-gapped network so poulling code over is often not possible. Thank! Regards

When calls to WSSendBinaryStream happen, I'd like to verify the contents went and are not buffered. If even does buffer (for the lack of a more precise term). if WComms.IsConnected WSSendBinaryStream(nil,Outdataa); I have events for OnSocketError and OnBGException and onWSFrameSent, checking AFrame.bytes. I am sending 3 4M files back to back. It looks as though all files were sent but after the first file sends and fires the on onWSFrameSent event, the last two files don't fire the event but I don' see any errors. Its like they are buffered or staged. If I send them again, the previous 2 files will send and fire the event, but it leaves 2 more either buffered or??? Sometimes all 3 files send, sometimes 2 etc..... But a fresh send will push the prefvious out and fire the onWSFrameSent. How can I verify all 3 frames were sent and if not, flush any remaining or resend? Regards Update: If I send the files one at a time, they transfer fine. In a loop, sending back-to-back seems to be the issue.

I was sifting through the source looking for examples of this very issue. I found MsgWaitForMultipleObjects right after you posted. I had tried sleep(0) with no success. But the MsgWaitForMultipleObjects worked and now the CPU is back down to .3 % and file transfers are back to their normal fast speed. Thanks!

Using OverbyteIcsSnippets as an example. Websocket CLIent section, after wsconnect is successful it falls in to this loop. while NOT AbortFlag do begin Application.ProcessMessages; if Application.Terminated then Break; if IcsTestTrgTick64(TrgTimerEnd) then Break; end; Mine is a console app and I do something almost identicle to this on startup. I connect then fall in to the main thread loop waiting for a signal to end. My application runs at 48% cpu usage just like the example. I realize this is a tight loop. Running as-is the OnWSFrameRecv will receive a 50M file in a few seconds. However when I try to address the high cpu issue and add a simple sleep(5), which drops the CPU down to less than 5 percent. The 50M file receive via the OnWSFrameRecv event now slows down to 9 minutes. Its like the receive is blocking. I'm not sure why a simple sleep would do this. Is the sleep cascading in the tight loop and causing the overall main thread to sleep much longer than the 5ms maybe? I know there is missing info here. The application is quite large but the websocket connect and receive is very basic. I rebuilt excluding throttling in the library but it didn't make any difference. Is there a better way this not CPU intensive to loop and wait for the termination? The goal is to receive large files via WS and not have the appl;ication idle at this high usage. Any help would be appreciated. Regards

I updated the above code hopefully for a little more clarity. So let me ask this a different way. In the above example WSConnect reaches the Server. The Server responds. WSConnect being a blocking call just hangs. IE no Windows messaging seems to be happening. I understanding writing a message pump, but ICS has both ProcessMessages and MessgePump build in to the library. So I am attempting to use those. OverbyteIcsSnippets1.pas does not address this as it uses Application.ProcessMessages which is not available to a DLL. I have tried various ways and places to insert ProcessMessages and MessagePump as you can see above with no luck. OnWSConnected event does not not run OnWSFrameRcvd event does not run What is the proper way to use the ICS MessagePump so t hat I can properly process messages throughout the DLL? Again, the console app works perfectly. When I switch it from a console app to DLL, WSConnect blocks and never returns, because I believe Messages aren't being processed.

Yes, but that doesn't answer the events question. How do I get events to process in a connection loop when the events don't fire. And WSConnect is the method I'm using. That loop was off the top of my head. Meant to illustrate a connection attempt every few seconds until it was connected without timing out. I'd pull over the actual code but I work on an air-gapped network so, I can't.

First, is there specific settings when running as a DLL? What is the right way to configure the component to run as DLL? I realize this is an event driven component but I can't seem to figure out how to get the events to fire properly when not a EXE. I have a test console app(exe) and it works great doing a wss connect to the host. When building and running the DLL, after the Connect it pauses for quite some time. May be a minute or so. The onConnect event doesn't seem to fire even though the server sees the inbound connection. The Connect is in a While NOT connected. Looks something like this procedure WSComms.ConnectHost; var Retries: integer; begin Retries:= 0; WSComms.WSPingSecs := 5; WSComms.Timeou t:= 0; WSComms.ResponseNoExceptions := True; WSComms.url:= 'wss:\\192.168.1.30' While (NOT WSComms.Connected) do begin sleep(2000); WSComms.ProcessMessages; //server sees the connectoin and responds but messages/events don't fire in the DL only WSComms.Connect; WSComms.MessagePump; Inc(Retries); If Retries > 5 then Exit; end; end; procedure WSDoconnection; begin //attempt connectoin until connected WSComms.ConnectHost; While (NOT AbortFlag) do begin WSComms.ProcessMessages; end; end; So this should attempt a connection around every 2 seconds or so..... But it stays on Connect for a good minute or more and doesn't actually connect. Again, the exe version works perfectly.... So its an events issue.

Thank you! Headed over now!

Yes. X509PubKeyTB Was the result of a previous thread here where I was doing the same validation with the ICS HTTPS Client. I added the code to pull the Mod(N) from the public key. which somehow became the standard in our shop to validate self signed certs. I ran in to this exact issue there not being able to use OnVerifyPeer as a final validation. This time around I'll opt for a more simple validation like described above. I did try to validate in the OnHandshakeDone but no luck with the OK flag overriding the chain validation. In OnVerifyPeer, if the OK flag would override any internal chain validation, that would work. Its ugly logic on our part. Its like.... I don't care what the cert validation chain says, If I say its valid, its valid. The Delphi NetHTTPClient uses this exact logic in their OnVerifyPeer override, which works great. If its assigned, all validation relies on the OnVerifyPeer OK flag. Also, I would think the purpose of having the VerifyPeer would be this exact case and solve RFC7250. Maybe? I'm going to press forward with my development because ICS WebSockets is very feature-rich and I'll need some of those features. Plus, I have faith you'll present a solution soon 🙂 Regards

Angus Using TSslWebSocketCli, after setting CertifyMethod to CertVerNone, verifypeer no longer gets executed. But the Websocket connects and is ready. If it completely bypasses all internal chain verification AND does not fire verifypeer (which it seems to behave the way you described), how do I do a manual cert verification? These are self signed certs and we validate using hashes, serialnum, pubkey(Mod N) etc... whatever the cert provides that is unique. Setting the CertifyMethod to CertVerBundle or CertVerdWinstore fires verifypeer but the certificates fail regardless of the _OK_ flag. I'm looking for something that may be described as CertVerSelf. This would fire the verifypeer to be used without other verification and the _OK_ would signify pass/fail of the cert. Is there a method in place to accomplish this already?

Thanks Angus. I traced the GET and its required during the http upgrade to WS. Thanks for pointing that out. Messagepump after the calls did the trick! Setting the following got me to a point where VerifyPeer is is being called and I can validate the cert. SslVerifyPeer:= MySslVerifyPeer; sslcontext.SslVerify:= True; SslAllowSelfSigned:= True; CertVerifyMethod:= CertVerBundle; <---- T his works, but is this the correct verification setting for ignoring cert validation and using VerifyPeer? I'm going through more examples today. Is there an example that shows data placed in a buffer and be sent when the socket is writable? LWS has a callback that pushes data whenever there is data in the buffer and the socket state is writeable. I'm still looking through examples 🙂 I'm not sure I understand how this works since TSslWebSocketCli is still needing the TSslContext to be created. Thanks again for supporting newbies!

Using TSslWebSocketCli, .messagepump did not make any difference. Also in the VCL app, after setting the url - 'wss://192.168.100.40:443/ws/data' it connects but issues an additional GET with an error reported on my server "GET /ws/data HTTP/1.1" error 404, for some sort of synchronization? This component seems to be geared toward http(s) rest GET PUT etc instead of straight forward socket ssl connections via a url and read/write to the socket. I also noticed OnStateChange no longer passes the previous and current state during state change. That was handy but not absolutely required IMO. I think I'm missing the correct usage. All I want to do is a WS SSL connection either via url or IP , PORT and path (preferably url), validate my cert and read/write data to and from the socket. All in a console app.

OK switched to TSslWebSocketCli. This component seems much more suitable. Lots of extra options... and the url setting works perfectly for concatenating my WS path together. This app is doing its own verifypeer with self signed certs. I'm missing a setting because I cant seem to get verifypeer event to fire using TSslWebSocketCli. It does using TSSLWSocket. How do I reference the automatically generated sslcontext you mentioned? or am I misunderstanding. Without assigning sslcontext, I cant set the SslContext.VerifyPeer:= True. It seems to want me to add/assign the sslcontext. I need to disable all automatic validation and rely on the OK check in the verifypeer proc. Plus I'm suffering the same issue with events working in my test VCL app but not being called in the console app. Do I just call the components .messagepump method before the wsconnect? Going to test that in a few.... One issue I have to yet investigate. Can I queue the data and when the socket becomes writable - send it? If not, how do I use TSslWebSocketCli to determine if the socket is ready/writable? Last, my apologies for not dropping code here. My dev network is air-gapped so I'd have to type it in a second time. It'll take a bit but I can if you say you need to see it.

My apologies for the confusion. Yes Websocket as a protocol. I understand that. I was originally using OverByteSimpleSslCli as an example reference and it uses TSslWSocket and TSslContext and it seemed like a great starting point. So I just now switched to TSslWebSocketCli and can do a little refactoring. But in regards to a console example and implementing events would be very helpful. I'll for sure checkout the Snippets. Thanks Angus.