Commented out exceptions in OverbyteIcsHttpPort

[sfrazor 3]

Commented out exception handling around line 3996.  

    {* if FStatusCode > 401 then    Dec 14, 2004 *}
//    if NOT FResponseNoException then begin  { V8.54 should error exceptions be skipped? }
//        if FRequestDoneError <> httperrNoError then                       { V7.23 }
//            raise EHttpException.Create(FReasonPhrase, FRequestDoneError) { V7.23 }
//        else if (FStatusCode >= 400) and (FStatusCode <> 401) and (FStatusCode <> 407) then
//            raise EHttpException.Create(FReasonPhrase, FStatusCode);
//    end;

 

Why?  I'm using Fr0sT-Brutal's memorymodule to load a DLL plugin.  When a parent DLL, loaded with loadlibrary, loads a child DLL via memorymodule, the exception handlers do not fire and a raise will kill the child (plugin) DLL when a connection failure happens.  Seems as though the try/except never fires correctly.  In original development I was using the exception handler to rotate dest IP's from Host1 to  Host2.

 

Using rundll32 to test the plugin DLL and building the DLL as an EXE everything works as expected.  SO its a combination of a DLL loading DLL via memorymodule.

 

Some background:  We have a DOS app that is a small loader.  It loads the main DLL via a normal loadlibrary which is the application itself.  The main app runs fine and all exceptions fire correctly.  However using memorymodule to load a child (plugin) DLL the try/excepts no longer work like I would expect.  I'd like to understand why.  After reading, I see that raising exceptions and something about DLL boundries will cause this problem?  I don't pretend to understand.

I originally  hit this roadblock when using TNETHTTPClient and a raise in System.Net.HTTPClient.Win line 1538.  Thiis was the original component used.  Using that component I was attemping to use the exception handler ot switch between Host1 and Host2 should a comms failure happen.  Before I can trap any errors or validate any http results, that raise kills the Child and cascades to the parent.  Again, this is with memorymodule loading the child. 

 

So, on to ICS.  Hoping I can use the above  idea to switch Hosts on connectoin failure events I ran in to a similar problem and decided to comment out the above section of Exceptions as a trial.  Exceptoins no longer kill the DLL.  I don't know ICS very well so capturing the failure and rotating Hosts is still something I'm working on.  Any suggestions would help.

 

I'm fairly certain there is no easy way to fix this considering I'm stuck using memorymodule.  

 

Will the above lines commented out hurt?  I'm concerned that some event handler, someplace,  relies on them and I just caused my self problems.   Or is there a better way to switch/rotate hosts upon failure that won't trip exceptions?  

 

Also, the child DLL mentioned is nothing more than a HTTPS client using GET to check for updates from a rest server.  In the past it never had to rotate hosts.  

 

Thanks for taking the time read all of this.

 

example.7z

Edited August 4, 2023 by sfrazor

[Angus Robertson 574]

If you don't want to raise an exception for response errors, you simply set the property ResponseNoException true, you don't need to comment out any code. 

 

No idea why an exception would kill a DLL, that must be your design, not handling exceptions correctly.

 

Never heard of memorymodule, not sure what relevance it has here.

 

Your main issue, unless I've read this wrong, would appear to connecting to alternate servers on failure.  If an ICS socket fails to connect, or you time out a connection attempt early, it takes a few moments for the socket to close and be ready for another connection, just because the Close event is called does not mean the socket is ready after errors, TCP has various timeouts when making connections. 

 

So where you need to contact sequential servers after failure, you should use alternate THttpCli components.  I usually have an array of components, all using the same events, with Tag set, so you know which component caused the event.    I did this recent in a new ICS component TIcsDomainNameCache which uses an array of components derived from THttpCli. 

 

Angus

 

 

 

 

[Fr0sT.Brutal 900]

I answered you in PM, so adding more specific info regarding ICS here: it is async so you have to setup message pump by yourself. I'm not sure ICS is the best choice here. Moreover, you have main DLL that loads child DLL via system's low-level functions (memorymodule) that itself loads SSL DLLs via regular LoadLibrary. It's just a nighmare Not the architecture I'd like to support

If you only target Windows, why not try WinInet/WinHTTP - the short sync function will do all the network magic for you, without any exceptions (result code is returned). Moreover it will respect proxy settings of a target OS.

[sfrazor 3]

On 8/5/2023 at 5:42 AM, Angus Robertson said:

If you don't want to raise an exception for response errors, you simply set the property ResponseNoException true, you don't need to comment out any code. 

 

No idea why an exception would kill a DLL, that must be your design, not handling exceptions correctly.

 

Never heard of memorymodule, not sure what relevance it has here.

 

Your main issue, unless I've read this wrong, would appear to connecting to alternate servers on failure.  If an ICS socket fails to connect, or you time out a connection attempt early, it takes a few moments for the socket to close and be ready for another connection, just because the Close event is called does not mean the socket is ready after errors, TCP has various timeouts when making connections. 

 

So where you need to contact sequential servers after failure, you should use alternate THttpCli components.  I usually have an array of components, all using the same events, with Tag set, so you know which component caused the event.    I did this recent in a new ICS component TIcsDomainNameCache which uses an array of components derived from THttpCli. 

 

Angus

 

 

 

 

Thanks Angus,

 

Never heard of memorymodule, not sure what relevance it has here.

Correct, Memorymodule has no relevance her except to expose all that I was trying to attempt and bring attention to what it was I was trying to do,  and it wasn't a simple loadmodule call.
 

 

The exacting issue is TNHTTPClient.Post and TNETHTTP.Get use exceptions to flag callbacks such as ValidateServerCertificate.  TNETHTTPCLient.Post generates an exception and then call back to ValidateServerCerrtivicate.  Normal loadmodule handles the exception and the validation callback happens fine.  All exceptions percolate to the top and the finally handles it.  That very same exception using memorymodule fails to call the validation callback and the raise kills the DLL. Hence I was looking at ICS and you mentioned ResponseNoException  property.  This is what I'm experimenting with now.

 

There seems to be no other components/libraries etc available to load a DLL from a resource file or stream.  In C, the public available memorymodule that Fr0sT.Brutal ported works perfectly in C and in Delphi (as long as no execptions are raised).

 

Now that I'm aware of the property for ResponseNoException  (I feel stupid for missing that). no commented line changes are needed 🙂  Thank you for that hint.

 

So to sum it up, I need to load DLL's from a resource file or memorystrream and manage HTTP Get and HTTP Put (HTTPS) talking to a REST API .  Something light-weight is preferred. I'm still looking for that.

[sfrazor 3]

On 8/7/2023 at 7:11 AM, Fr0sT.Brutal said:

I answered you in PM, so adding more specific info regarding ICS here: it is async so you have to setup message pump by yourself. I'm not sure ICS is the best choice here. Moreover, you have main DLL that loads child DLL via system's low-level functions (memorymodule) that itself loads SSL DLLs via regular LoadLibrary. It's just a nighmare Not the architecture I'd like to support

If you only target Windows, why not try WinInet/WinHTTP - the short sync function will do all the network magic for you, without any exceptions (result code is returned). Moreover it will respect proxy settings of a target OS.

Thanks Frost, I'm using TNETHTTPClient - which is just a WINHttp wrapper it seems.  It seems pretty fast and fairly lightweight that handles HTTPS get and post and allows for cert verification and doesn't generate callbacks based on raising exceptions.  With the added bonus of cross platform and no openssl dlls needed.

 

memorymodule works pretty well.  Not sure what's wrong that exceptions won't percolate to the top.  The C version works because callbacks are pointers set intentionally and of coarse there are no exceptions to muck thing up.  Its all result codes.  If there were a lightweight HTTPS put and get (preferably that don't need openssl...  I use YuOpenSSL when I can) that don't use exception handlers to manage traffic memorymodule would work.  

 

As I mentioned to Angus, I don't see any good options for loading DLL's from resource or streams.

 

I set the code aside and am looking at ICS today and tomorrow.  WAY more features than I need.

 

I really appreciate everyone here taking the time to try and help.

 

 

[Fr0sT.Brutal 900]

8 hours ago, sfrazor said:

I'm using TNETHTTPClient - which is just a WINHttp wrapper it seems

Yes but it raises exceptions while WinHttp doesn't. If multiplatform is not the goal, using WinAPI directly could help avoiding exceptions

 

As for exceptions, probably something in the way Delphi handles them causes the issue. You'll probably face it again when exception will be unavoidable. I wouldn't recommend using memmodule approach in production with such a problem unsolved. Do you really really need to load DLLs without saving them to a file?

Btw, are you using master or dev branch? dev had much updates but is untested.