sfrazor

First, is there specific settings when running as a DLL? What is the right way to configure the component to run as DLL? I realize this is an event driven component but I can't seem to figure out how to get the events to fire properly when not a EXE. I have a test console app(exe) and it works great doing a wss connect to the host. When building and running the DLL, after the Connect it pauses for quite some time. May be a minute or so. The onConnect event doesn't seem to fire even though the server sees the inbound connection. The Connect is in a While NOT connected. Looks something like this procedure WSComms.ConnectHost; var Retries: integer; begin Retries:= 0; WSComms.WSPingSecs := 5; WSComms.Timeou t:= 0; WSComms.ResponseNoExceptions := True; WSComms.url:= 'wss:\\192.168.1.30' While (NOT WSComms.Connected) do begin sleep(2000); WSComms.ProcessMessages; //server sees the connectoin and responds but messages/events don't fire in the DL only WSComms.Connect; WSComms.MessagePump; Inc(Retries); If Retries > 5 then Exit; end; end; procedure WSDoconnection; begin //attempt connectoin until connected WSComms.ConnectHost; While (NOT AbortFlag) do begin WSComms.ProcessMessages; end; end; So this should attempt a connection around every 2 seconds or so..... But it stays on Connect for a good minute or more and doesn't actually connect. Again, the exe version works perfectly.... So its an events issue.

I updated the above code hopefully for a little more clarity. So let me ask this a different way. In the above example WSConnect reaches the Server. The Server responds. WSConnect being a blocking call just hangs. IE no Windows messaging seems to be happening. I understanding writing a message pump, but ICS has both ProcessMessages and MessgePump build in to the library. So I am attempting to use those. OverbyteIcsSnippets1.pas does not address this as it uses Application.ProcessMessages which is not available to a DLL. I have tried various ways and places to insert ProcessMessages and MessagePump as you can see above with no luck. OnWSConnected event does not not run OnWSFrameRcvd event does not run What is the proper way to use the ICS MessagePump so t hat I can properly process messages throughout the DLL? Again, the console app works perfectly. When I switch it from a console app to DLL, WSConnect blocks and never returns, because I believe Messages aren't being processed.

Yes, but that doesn't answer the events question. How do I get events to process in a connection loop when the events don't fire. And WSConnect is the method I'm using. That loop was off the top of my head. Meant to illustrate a connection attempt every few seconds until it was connected without timing out. I'd pull over the actual code but I work on an air-gapped network so, I can't.

Some background. I'm replacing a C console WS app with a Delphi console app, eventually a DLL. The C WS is libwebsockets. Using ICS, SSL (YuOpenSSL), TSslcontext and TSSLWSocket. in the constructor, I want to assign class procedures for items such as listed below. constructor TMyWSComms.Create(AOwner: TComponent); begin inherited create(AOwner); Sslcontext:= TSslCOntext.create; SslEnable:= True; Sslcontext.SslVerifyPeer:= True; addr:= 192.168.100.40; port:= 443; // overrides with class methods OnSessionConnected:= MyOnSessonConnected; OnSessionClosed:= MyOnSessonClosed; OnStateChanged:= MyOnStateChanged; OnSslVerifyPeer:= MySslVerifyPeer; // several more..... end Using ICS as VCL this all works fine and events fire as expected. But not as a console app, It seems I'm missing a message pump? With ICS I'm running in to a couple of issues: First, there are built in message pumps available in ICS but I'm not sure how to implement that or if that is even what I need. If there is an example of this, please point me to it. Second, Even though I Assign/create a SslContext I get an error on socket shutdown inside ICS saying Sslcontext is required/missing. IJ can provide the source and line number in ICS if needed. Last, with libwebsockets there are 3 params to set for a connection (more if you include the socket and ssl setup) address port path So if I want to connect to 192.168.100.40:443/ws/interface I set the above accordingly: address:= 192.168.100.40 port: 443 path '/ws/interface' where /ws/interface/ is an expected python tornado connection path. How do I construct this using ICS? Sounds trivial but I'm completely missing it. But with ICS there is no path equivalent that I see.

Thank you! Headed over now!

Yes. X509PubKeyTB Was the result of a previous thread here where I was doing the same validation with the ICS HTTPS Client. I added the code to pull the Mod(N) from the public key. which somehow became the standard in our shop to validate self signed certs. I ran in to this exact issue there not being able to use OnVerifyPeer as a final validation. This time around I'll opt for a more simple validation like described above. I did try to validate in the OnHandshakeDone but no luck with the OK flag overriding the chain validation. In OnVerifyPeer, if the OK flag would override any internal chain validation, that would work. Its ugly logic on our part. Its like.... I don't care what the cert validation chain says, If I say its valid, its valid. The Delphi NetHTTPClient uses this exact logic in their OnVerifyPeer override, which works great. If its assigned, all validation relies on the OnVerifyPeer OK flag. Also, I would think the purpose of having the VerifyPeer would be this exact case and solve RFC7250. Maybe? I'm going to press forward with my development because ICS WebSockets is very feature-rich and I'll need some of those features. Plus, I have faith you'll present a solution soon 🙂 Regards

Angus Using TSslWebSocketCli, after setting CertifyMethod to CertVerNone, verifypeer no longer gets executed. But the Websocket connects and is ready. If it completely bypasses all internal chain verification AND does not fire verifypeer (which it seems to behave the way you described), how do I do a manual cert verification? These are self signed certs and we validate using hashes, serialnum, pubkey(Mod N) etc... whatever the cert provides that is unique. Setting the CertifyMethod to CertVerBundle or CertVerdWinstore fires verifypeer but the certificates fail regardless of the _OK_ flag. I'm looking for something that may be described as CertVerSelf. This would fire the verifypeer to be used without other verification and the _OK_ would signify pass/fail of the cert. Is there a method in place to accomplish this already?

Thanks Angus. I traced the GET and its required during the http upgrade to WS. Thanks for pointing that out. Messagepump after the calls did the trick! Setting the following got me to a point where VerifyPeer is is being called and I can validate the cert. SslVerifyPeer:= MySslVerifyPeer; sslcontext.SslVerify:= True; SslAllowSelfSigned:= True; CertVerifyMethod:= CertVerBundle; <---- T his works, but is this the correct verification setting for ignoring cert validation and using VerifyPeer? I'm going through more examples today. Is there an example that shows data placed in a buffer and be sent when the socket is writable? LWS has a callback that pushes data whenever there is data in the buffer and the socket state is writeable. I'm still looking through examples 🙂 I'm not sure I understand how this works since TSslWebSocketCli is still needing the TSslContext to be created. Thanks again for supporting newbies!

Using TSslWebSocketCli, .messagepump did not make any difference. Also in the VCL app, after setting the url - 'wss://192.168.100.40:443/ws/data' it connects but issues an additional GET with an error reported on my server "GET /ws/data HTTP/1.1" error 404, for some sort of synchronization? This component seems to be geared toward http(s) rest GET PUT etc instead of straight forward socket ssl connections via a url and read/write to the socket. I also noticed OnStateChange no longer passes the previous and current state during state change. That was handy but not absolutely required IMO. I think I'm missing the correct usage. All I want to do is a WS SSL connection either via url or IP , PORT and path (preferably url), validate my cert and read/write data to and from the socket. All in a console app.

OK switched to TSslWebSocketCli. This component seems much more suitable. Lots of extra options... and the url setting works perfectly for concatenating my WS path together. This app is doing its own verifypeer with self signed certs. I'm missing a setting because I cant seem to get verifypeer event to fire using TSslWebSocketCli. It does using TSSLWSocket. How do I reference the automatically generated sslcontext you mentioned? or am I misunderstanding. Without assigning sslcontext, I cant set the SslContext.VerifyPeer:= True. It seems to want me to add/assign the sslcontext. I need to disable all automatic validation and rely on the OK check in the verifypeer proc. Plus I'm suffering the same issue with events working in my test VCL app but not being called in the console app. Do I just call the components .messagepump method before the wsconnect? Going to test that in a few.... One issue I have to yet investigate. Can I queue the data and when the socket becomes writable - send it? If not, how do I use TSslWebSocketCli to determine if the socket is ready/writable? Last, my apologies for not dropping code here. My dev network is air-gapped so I'd have to type it in a second time. It'll take a bit but I can if you say you need to see it.

My apologies for the confusion. Yes Websocket as a protocol. I understand that. I was originally using OverByteSimpleSslCli as an example reference and it uses TSslWSocket and TSslContext and it seemed like a great starting point. So I just now switched to TSslWebSocketCli and can do a little refactoring. But in regards to a console example and implementing events would be very helpful. I'll for sure checkout the Snippets. Thanks Angus.

You would have to ask public opinion on this one. Most would probably say 'why bother doing this at all'. To a limited few it would be a blessing. I would use it in my current project depending on how large adding the resource makes the DLL's overall. A couple of things should be noted: As just mentioned to me by Fr0sT.Brutal there is no auto-run. DLLMain calls are not supported that I am aware of. So if you are counting on DLLMain, OpenSSL wont initialize. Currently when I call SSL_CTX_new it fails ONLY when called from a memoryloaded DLL. This may be the root of a failure I'm currently experiencing. I need to test that theory and discuss with Ralf (YuOpenSSL). Someone please correct me if I'm wrong. Next, and I'm saying this without solid verification, exception handling in memorymodule loaded DLL's doesn't work maybe due to .SAFESEH section of the DLL not being mapped/loaded. If this is the case Windows won't allow SEH Handlers because it believes they are not safe. It will crash the app using memorymodule. This needs to be fixed. I'm trying to understand this part now so anyone wanting to correct me please do so. I am no authority on this. and last, BTMemorymodule is only 32bit last time I looked. Has that changed? Blackbone an Dark Memory Loader both correct the above issues in C but there is no equivalent in Delphi unless that is stashed away in private repos. I'm just not sure which one would be a viable repo to fork from.... So, Angus, proceed with caution 🙂

I have used both MemoryModuyle from Fr0sT.Brutal and BTMemoryModule from Martin Offenwanger linked above. Both were stable enough for our production code recognizing there are limitations and am adapting code using ICS with YuOpenSSL. We chose the MemoryModule from Fr0sT.Brutal only because it was the first one we came across and its fairly solid. If there is an improved branch of either I'd love to have it. I could go in to a lengthy description of an issue with both if anyone is interested. The issue is similar between the two. I use both with YuOpenSSL which works very well. I highly recommend it as a OpenSSL snap-in. Including using it with ICS and Synapse (I haven't had much luck using it with Indy). I think Ralf should open a 3rd party section here for his product FWIW. @FroSt.Brutal for an abandoned project, its pretty darn good. I pulled the dev branch and made a slight mod to remove the Ordinal entry points because it was giving me an error and I just didn't need it. Regardless of the nay-sayers asking why and how come etc... there is a need. In my case I have no choice as its in legacy code that I am forced to build upon. The only issue I have with either is figuring out how to debug a DLL loaded with these memory loaders. I'm learning but thats beyond my skill set. Can anyone help? @FroSt.Brutal check your PM please. Angus, either is an excellent starting point for your memory based resource loader and the need is there. IMO.

Thank you. managing certs is all magic to me. This is definitely a learning experience. So, decode and clean up the text... Its a fairly simple server in python (tornado). However. we currently have a ticket in our backlog to rewrite with additional features. It will be ICS at some point as we've been asked to move away from tornado/python and toward RAD Studio solutions I added your Raw Public Key TLS support to the comments in the ticket for consideration. All clients ( 2 and soon to be 3) are currently TNETHttpClient based dll's and as mentioned, failover has become a priority. As also noted above, the dll plugins are memory loaded and I'm stuck with that architecture for the time being. TNetHttpClient uses exceptions to determine connection failures and rollover causes the dll's to crash when an exception is hit/raised. I am working on a replacement that uses error codes in comms instead.

ICS 8.70 and Delphi 11.3 Using TNetHTTPClient there is an event handler that allows verification of key data and setting the accept to true/false. OnValidateServerCertiicate(const Sender: TObject; const ARequest: TURLRequest; const certificate; var Accepted: boolean). I am currently working with TSslHttpRest. I am unsure how to accomplish this same validation check in ICS. If there is a demo that illustrates this, or some direction, I would appreciate the feedback.

Thanks Angus. I had to have a look at our pipeline. Its producing a PEM formatted cert(s). We have a server and a couple of apps that are using it. I'll look in to changing to DER and see how far I get. I seem to recall being able to convert to-from PEM/DER. Not something I'm familiar with. I really appreciate the work!!

There is 2 servers on the network both using the same CA for keygen during the CI/CD pipeline builds. That process is how the client apps get the pubkey imbedded when they are built. Its a one-to-many between the server and app. No other servers besides the primary and failover servers will have that pubkey and the apps will only rollover to a secondary that has the cert signed by the same CA. This effort is a progression from a previous post where DLL plugins loaded with memorymodule cause bad results when exception handling is used for comms state instead of error codes during a failover. In the case NetHttpClient, that's a show-stopper now. ICS offers a way to disable that and rely on error codes as you pointed out in a previous post plus it supports YuOpenSSL. Thus the ICS investigation. I'd have to mod our pipeline builds to use a hash instead of publickey. That also means I'd have to redistribute the current 2 rest apps (soon to be 3) to validate against a hash. I'm still investigating pulling the X509PublicKey via the pointer that is passed in via peerverify.

2) Put that root certificate in place to accessed by the client, either install it in Windows store or even better use the ICS bundle store. Unfortunately this is not an option. Thanks for the input Kass Ob. I completely understand. If this were a normal client app I would do it exactly as you mentioned. In this case we're dealing with some imbedded code and size matters. Pushing the entire cert in the binary is not preferred and there is no place to write the pem(cert) to on the client. So in order to validate the cert belongs to a specific server, the publickey is imbedded and used for validation. We could imbed the key fingerprint instead. But none of the comms components I've seen surface the fingerprint.

Thank you Angus for looking in to this. Unfortunately it seems I'm stuck with TNETHttpClient since it surfaces all aspects of the key for manual validation including the public key. CertName SerialNum Subject Issuer ProtocolName AlgSignature AlgEncrytpion Cert Start Date Cert Exp Date Keysize publicKey (RSA Modulus (n) in hex) Seems all it is missing is fingerprint. When SSL has determined the keys are a problem and the callback allows override, publickey is very popular for manual validation because its small and easy to embed. I'll investigate pulling the X509PublicKey but I don't think I'll be allowed the time. Please consider surfacing publickey in the future. 🙂

unit Unit6; interface uses Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics, Vcl.Controls, Vcl.Forms, Vcl.Dialogs, Vcl.StdCtrls, OverbyteIcsWndControl, OverbyteIcsHttpProt, OverbyteIcsWSocket; type TForm6 = class(TForm) Memo1: TMemo; Button1: TButton; SslHttpCli1: TSslHttpCli; SslContext1: TSslContext; procedure Button1Click(Sender: TObject); procedure SslHttpCli1SslVerifyPeer(Sender: TObject; var Ok: Integer; Cert: TX509Base); procedure SslHttpCli1SslHandshakeDone(Sender: TObject; ErrCode: Word; PeerCert: TX509Base; var Disconnect: Boolean); private { Private declarations } public { Public declarations } end; var Form6: TForm6; implementation {$R *.dfm} procedure TForm6.Button1Click(Sender: TObject); begin SslHttpCli1.URL := 'https://www.google.com'; //choose any https server SslHttpCli1.ResponseNoException := True; SslHttpCli1.Get; end; procedure TForm6.SslHttpCli1SslHandshakeDone(Sender: TObject; ErrCode: Word; PeerCert: TX509Base; var Disconnect: Boolean); var MyKey: string; begin Memo1.Lines.Add('HandshakeDone: ' + IntToStr(ErrCode)); Memo1.Lines.Add(PeerCert.GetPKeyRawText(True)); if PeerCert.GetPKeyRawText(True) = MyKey then <<<-- this is wrong and is probably the private key Disconnect := False else Disconnect := True; end; procedure TForm6.SslHttpCli1SslVerifyPeer(Sender: TObject; var Ok: Integer; Cert: TX509Base); var MyKey: string; begin Memo1.Lines.Add('VerifyPeer'); end; end. I think the question is, in either VerifyPeer or HandshakeDone, how do I retrieve the peer pubkey for validation? A hex string would be optimal. Also in the above is GetPKeyRawText(True)); for retrieving the public key or private key?

Thank you. I'll look at that example. Working strictly with self signed keys in a closed environment, I need to specifically validate (compare) a compiled in public key in the client to the public Key sent by the server and accept or reject based on that compare.

Commented out exception handling around line 3996. {* if FStatusCode > 401 then Dec 14, 2004 *} // if NOT FResponseNoException then begin { V8.54 should error exceptions be skipped? } // if FRequestDoneError <> httperrNoError then { V7.23 } // raise EHttpException.Create(FReasonPhrase, FRequestDoneError) { V7.23 } // else if (FStatusCode >= 400) and (FStatusCode <> 401) and (FStatusCode <> 407) then // raise EHttpException.Create(FReasonPhrase, FStatusCode); // end; Why? I'm using Fr0sT-Brutal's memorymodule to load a DLL plugin. When a parent DLL, loaded with loadlibrary, loads a child DLL via memorymodule, the exception handlers do not fire and a raise will kill the child (plugin) DLL when a connection failure happens. Seems as though the try/except never fires correctly. In original development I was using the exception handler to rotate dest IP's from Host1 to Host2. Using rundll32 to test the plugin DLL and building the DLL as an EXE everything works as expected. SO its a combination of a DLL loading DLL via memorymodule. Some background: We have a DOS app that is a small loader. It loads the main DLL via a normal loadlibrary which is the application itself. The main app runs fine and all exceptions fire correctly. However using memorymodule to load a child (plugin) DLL the try/excepts no longer work like I would expect. I'd like to understand why. After reading, I see that raising exceptions and something about DLL boundries will cause this problem? I don't pretend to understand. I originally hit this roadblock when using TNETHTTPClient and a raise in System.Net.HTTPClient.Win line 1538. Thiis was the original component used. Using that component I was attemping to use the exception handler ot switch between Host1 and Host2 should a comms failure happen. Before I can trap any errors or validate any http results, that raise kills the Child and cascades to the parent. Again, this is with memorymodule loading the child. So, on to ICS. Hoping I can use the above idea to switch Hosts on connectoin failure events I ran in to a similar problem and decided to comment out the above section of Exceptions as a trial. Exceptoins no longer kill the DLL. I don't know ICS very well so capturing the failure and rotating Hosts is still something I'm working on. Any suggestions would help. I'm fairly certain there is no easy way to fix this considering I'm stuck using memorymodule. Will the above lines commented out hurt? I'm concerned that some event handler, someplace, relies on them and I just caused my self problems. Or is there a better way to switch/rotate hosts upon failure that won't trip exceptions? Also, the child DLL mentioned is nothing more than a HTTPS client using GET to check for updates from a rest server. In the past it never had to rotate hosts. Thanks for taking the time read all of this. example.7z

Thanks Frost, I'm using TNETHTTPClient - which is just a WINHttp wrapper it seems. It seems pretty fast and fairly lightweight that handles HTTPS get and post and allows for cert verification and doesn't generate callbacks based on raising exceptions. With the added bonus of cross platform and no openssl dlls needed. memorymodule works pretty well. Not sure what's wrong that exceptions won't percolate to the top. The C version works because callbacks are pointers set intentionally and of coarse there are no exceptions to muck thing up. Its all result codes. If there were a lightweight HTTPS put and get (preferably that don't need openssl... I use YuOpenSSL when I can) that don't use exception handlers to manage traffic memorymodule would work. As I mentioned to Angus, I don't see any good options for loading DLL's from resource or streams. I set the code aside and am looking at ICS today and tomorrow. WAY more features than I need. I really appreciate everyone here taking the time to try and help.

Thanks Angus, Never heard of memorymodule, not sure what relevance it has here. Correct, Memorymodule has no relevance her except to expose all that I was trying to attempt and bring attention to what it was I was trying to do, and it wasn't a simple loadmodule call. The exacting issue is TNHTTPClient.Post and TNETHTTP.Get use exceptions to flag callbacks such as ValidateServerCertificate. TNETHTTPCLient.Post generates an exception and then call back to ValidateServerCerrtivicate. Normal loadmodule handles the exception and the validation callback happens fine. All exceptions percolate to the top and the finally handles it. That very same exception using memorymodule fails to call the validation callback and the raise kills the DLL. Hence I was looking at ICS and you mentioned ResponseNoException property. This is what I'm experimenting with now. There seems to be no other components/libraries etc available to load a DLL from a resource file or stream. In C, the public available memorymodule that Fr0sT.Brutal ported works perfectly in C and in Delphi (as long as no execptions are raised). Now that I'm aware of the property for ResponseNoException (I feel stupid for missing that). no commented line changes are needed 🙂 Thank you for that hint. So to sum it up, I need to load DLL's from a resource file or memorystrream and manage HTTP Get and HTTP Put (HTTPS) talking to a REST API . Something light-weight is preferred. I'm still looking for that.

Spending time on a function to ping continuously I noticed that I can't seem to kill the proccess I started. I googled around a bit and copied this peice of code to learn with and noticed that if a command ends properly all is well. I thought I knew how and I've tried a few things. You'll notice the lines commented out of the various things I tried. The end goal is to collect the ping results until a time expires. But I never got that far. In task manager, after each of he attempts to terminate the ping, it shows to be still running. What am I doing wrong? procedure GetDosOutput; var SA: TSecurityAttributes; SI: TStartupInfo; PI: TProcessInformation; StdOutPipeRead, StdOutPipeWrite: THandle; WasOK: Boolean; Buffer: PAnsiChar; BytesRead: Cardinal; WorkDir: string; Handle: Boolean; CommandLine: string; Output: String; taskkill: string; begin CommandLine:='c:\windows\system32\cmd.exe /c ping -t 127.0.0.1'; Buffer:=AllocMem(128000); Output := ''; with SA do begin nLength := SizeOf(SA); bInheritHandle := True; lpSecurityDescriptor := nil; end; CreatePipe(StdOutPipeRead, StdOutPipeWrite, @SA, 0); try with SI do begin FillChar(SI, SizeOf(SI), 0); cb := SizeOf(SI); dwFlags := STARTF_USESHOWWINDOW or STARTF_USESTDHANDLES; wShowWindow := SW_HIDE; hStdInput := GetStdHandle(STD_INPUT_HANDLE); hStdOutput := StdOutPipeWrite; hStdError := StdOutPipeWrite; end; PI:= Default(TProcessInformation); UniqueString(CommandLine); Handle := CreateProcess(nil, PWIdeChar(CommandLine), nil, nil, True, 0, nil, nil, SI, PI); CloseHandle(StdOutPipeWrite); if Handle then try repeat WasOK := ReadFile(StdOutPipeRead, Buffer, 128000, BytesRead, nil); if BytesRead > 0 then begin Buffer[BytesRead] := #0; Output := Output + Buffer; end; until not WasOK or (BytesRead = 0); WaitForSingleObject(PI.hProcess, 5000); finally //TerminateProcess(PI.hProcess,0); //CloseHandle(PI.hThread); //CloseHandle(PI.hProcess); end; finally CloseHandle(StdOutPipeRead); end; // taskkill:= '/C taskkill /f /t /PID = '+ INtToStr(PI.dwProcessId); // ShellExecute(0, nil, 'cmd.exe',pWideChar(taskkill), nil, SW_HIDE); // taskkill:= '/C taskkill /f /t /PID = '+ INtToStr(PI.hProcess); // ShellExecute(0, nil, 'cmd.exe',pWideChar(taskkill), nil, SW_HIDE); FreeMem(Buffer); end;