ICS SLL3.2 much slower than Indy SSL1.0.2

[PizzaProgram 9]

I've finally finished to rework all my codes, so it works both with old Indy OpenSSL 1.0.2

and latest ICS 9.1 (OpenSSL 3.2) by setting a simple checkbox (UseSSL3?)

 

But when I test it, it gives odd results, because it takes 3x so much time to download a simple XML with ICS now.

 

Old Indy download time = 560 - 570ms

New ICS download time = 1560-1580ms

 

I have tested it many many times, same results.

- Both components running in background thread.

- BasicAuth (username + password) is used.

 

- Indy component is getting created each time before download. But ICS (+SSL initialization) is done only once before thread start.

 

The code I use is very simple:

function Get_with_SSL3(const url: string; SslHttpCli3: TSslHttpCli; var Data:TMemoryStream; var error: string; timeOut: integer): Boolean;
var
    size: Int64;
begin
    Result := False;
    if Data = nil then
        Data    := TMemoryStream.Create;

    SslHttpCli3.URL := url;
    SslHttpCli3.Timeout := timeOut;
    try
        try
            SslHttpCli3.Get;
        except on E:Exception do  // 400 
            error := E.Message;
        end;
    finally
        Result := (SslHttpCli3.StatusCode = 200) or (SslHttpCli3.StatusCode = 400);
        size := 0;
        try
            if Assigned(SslHttpCli3.RcvdStream) then begin
                SslHttpCli3.RcvdStream.Seek(0, soFromBeginning);
                size := SslHttpCli3.RcvdStream.Size;
            end;
            if size > 0 then begin
                Data.Clear;
                Data.LoadFromStream( SslHttpCli3.RcvdStream );
            end
            else
                error := 'ERROR! 0 byte answer. ' + error;
        except on E: Exception do
            hiba := '! Stream read error: ' + CRLF + E.Message;
        end;

        if Assigned(SslHttpCli3.SendStream) then SslHttpCli3.SendStream.Size := 0;
        if Assigned(SslHttpCli3.RcvdStream) then SslHttpCli3.RcvdStream.Size := 0;
    end;
end;

 

Is this "normal" ?

Edited April 20 by PizzaProgram

[Angus Robertson 574]

You are concerned about a one second longer download on a tiny file?   With different SSL protocols and ciphers. 

 

Angus

 

[FPiette 382]

28 minutes ago, PizzaProgram said:

But ICS (+SSL initialization) is done only once before thread start.

Doing like this is likely makes your ICS component in the main thread context.

You should create the ICS component inside the thread execute method.

29 minutes ago, PizzaProgram said:

old Indy OpenSSL 1.0.2 and latest ICS 9.1 (OpenSSL 3.2)

You cannot compare with different OpenSSL versions.

 

Best performance with ICS is when using asynchronous operation and events. Synchronous operation will slow things down. You probably not even need a thread in the first place!

[Remy Lebeau 1393]

Quote

Indy component is getting created each time before download.

Why? You can reuse TIdHTTP for multiple requests.

Quote

But ICS (+SSL initialization) is done only once before thread start.

Are you creating it in the thread constructor? Or inside of the thread Execute()? They run in different thread contexts

[PizzaProgram 9]

Thanks for all the answers !

 

To clarify things:

1.  Yes, I create those components in the background threads. Each thread = each separate http component. 👼
2.  No, it would be a big mistake to put everything in the main thread. 🤯
    This is a POS program with a max user-response time of 50-100ms of the Main thread.
    (And there are 4-5 background threads doing different things, like DB-locking, DB-searching, DB-executing+ commiting + order cashing, menu-syncing, Logging, CallerID, Syslog, etc. So blocking the main thread would be more than problematic.)

Seriously, I do not try to blame anybody, both components are excellent works ! Congrat to both of you and I'm really glad I can still use my old delphi7 for such complex things, like web-server + https.

But that time increase from 580ms to 1560 is not normal, and I'd like to dig to the bottom of it.

 

•  - Is there something maybe misconfigured?
•  - Why does the 2th, 3th, etc call of ICS taking the same time as the 1th?
    I do not destroy the component, do not tell it to disconnect, so the socket should be established to the same domain, only some sub-parameters of the URL is changing !
   So theoretically the speed should shorten after the first connection.
•  - Opening the URL in a browser: Firefox time measure shows = 560ms too, so that is the standard server response time.
•  - (Ping to the server is constant 2ms)

 

[FPiette 382]

2 hours ago, PizzaProgram said:

Yes, I create those components in the background threads.

You are not clear enough. You MUST create ICS component in the thread's Execute() method. Creating it in the thread's constructor will result of all events running in the main thread context.

2 hours ago, PizzaProgram said:

it would be a big mistake to put everything in the main thread. 🤯
 This is a POS program with a max user-response time of 50-100ms of the Main thread.

The mistake is probably having the main thread with a less than 100mS response time. Are you polling devices from the main thread? If yes, that's there you have to move code to a thread.

[PizzaProgram 9]

5 hours ago, FPiette said:

You MUST create ICS component in the thread's Execute() method. Creating it in the thread's constructor will result of all events running in the main thread context.

Well, I'm shocked. You were right!

I've moved the creation of the components from the initialization part to the Execute procedure and It's the same fast now as Indy.

 

Also this will probably explain WHY my application crashed always ... (link to prev. topic about this problem)

 

Thank you very much for the help!

I truly recommend, the future versions of the component should check, and raise an error if:

1. The creation and running thread are not the same?
2. The first initialization (of loading SSL DLLs) and final destruction is not running in the main thread!
3. Property of .Multithreaded is True while running in a background thread? (Can be automated too.)

It's not just me, who recommended that:

On 8/21/2023 at 12:21 PM, Dalija Prasnikar said:

Those few cycles mean nothing comparing to constructing the whole component over and over again. So this kind of safeguard can be easily implemented without having any negative side effects. The property itself could be left as-is, but initial value can be set in constructor depending on the thread ID value:

FMultiThreaded := TThread.CurrentThread.ThreadID <> MainThreadID;

 

 

[PizzaProgram 9]

What I still don't know:

• Should I set myGlobalMainCLIcomponent.MultiThreaded := TRUE; for the Main thread's component too ?
• Or only for those running in the background thread ?

[FPiette 382]

11 hours ago, PizzaProgram said:

17 hours ago, FPiette said:

You MUST create ICS component in the thread's Execute() method. Creating it in the thread's constructor will result of all events running in the main thread context.

Well, I'm shocked. You were right!

I've moved the creation of the components from the initialization part to the Execute procedure and It's the same fast now as Indy.

Your ICS component event wre handled by the main thread which - you said that - is quite busy. Now that you created to component in thread's execute method, the events are handled by the thread. This is how Windows work with asynchronous operation (cooperative multitasking).

 

11 hours ago, PizzaProgram said:

I truly recommend, the future versions of the component should check, and raise an error if:

1. The creation and running thread are not the same?
2. The first initialization (of loading SSL DLLs) and final destruction is not running in the main thread!
3. Property of .Multithreaded is True while running in a background thread? (Can be automated too.)

We cannot change the component for you own need. You have the tools to do that in your own code. In the vast majority of programs using ICS component, no multithreading is ever needed. You have something wrong in your main thread for it to be so slow. Check your design and place multithreading where it is really helpful.

 

I have an application using ICS that has more than 700 active users and yet doesn't make use of any thread. Both client and server are using ICS.

[PizzaProgram 9]

On 4/21/2024 at 10:28 PM, PizzaProgram said:

I truly recommend, the future versions of the component should check, and raise an error if:

1. The creation and running thread are not the same?
2. The first initialization (of loading SSL DLLs) and final destruction is not running in the main thread!
3. Property of .Multithreaded is True while running in a background thread? (Can be automated too.)

 

I would like to report once again:

 - Since the last 14 days NO MORE PROGRAM ERRORS occurred any more caused by ICS running in background thread .

 - running on 100+ PCs

 

I strongly suggest implementing those 3 fail-safe lines of codes at the next release, to prevent such behaviour !

 (Because it took me 8+ month and many gray hair to find out what's causing it and how to fix it. Also there is no written warning about these anywhere.)