How to know when TIdHTTPServer threads are done

[narag 0]

Hi,

 

I'm using a TIdHTTPServer component and I would like to know, after its Active property is set to False, when all the current threads have finished serving. 

 

I could hold a count of active threads, but maybe instead of introducing just another synchronization mechanism, there's some out of the box property or event that I could use.

 

TIA,

 

   Nico

[Lajos Juhász 311]

 

5 minutes ago, narag said:

I'm using a TIdHTTPServer component and I would like to know, after its Active property is set to False, when all the current threads have finished serving. 

 

It should be done as the SetActive returns. If You look at the source code SetActive will call Shutdown that will call TerminateAllThreads.

[narag 0]

Just now, Lajos Juhász said:

 

 

It should be done as the SetActive returns. If You look at the source code SetActive will call Shutdown that will call TerminateAllThreads.

Thank you, that will do.

[narag 0]

Wait, so will Active := False shut up the current connections? I've followed the code and there's a closesocket in DoTerminateContext.

 

That's not very nice to the clients. My intention is to hot swap servers gracefully, without causing connections errors. Even StopListening seems to close threads' sockets.

 

If there's no "soft" way to stop accepting new connections, it seems I need to write that active thread list.

 

 

[Lajos Juhász 311]

At procedure TIdCustomTCPServer.Shutdown before the threads are terminated StopListening is called:

 

//APR-011207: for safe-close Ex: SQL Server ShutDown 1) stop listen 2) wait until all clients go out
 

[Remy Lebeau 1498]

2 hours ago, narag said:

I would like to know, after its Active property is set to False, when all the current threads have finished serving. 

Setting the Active property to False does a full server shutdown. It blocks the calling thread until all clients are fully disconnected and all threads have fully terminated.

1 hour ago, narag said:

Wait, so will Active := False shut up the current connections?

Yes.

1 hour ago, narag said:

That's not very nice to the clients.

Why? The whole point of setting the Active property to False is to shut down the server completely. 

1 hour ago, narag said:

My intention is to hot swap servers gracefully

Meaning what, exactly? What are you actually trying to accomplish? 

1 hour ago, narag said:

Even StopListening seems to close threads' sockets.

StopListening() merely closes the listening sockets and their threads. It does not close any clients or stop their threads. Closing a listening socket does not close any clients that connected to it, as those are independent sockets once they have been accepted.

1 hour ago, narag said:

If there's no "soft" way to stop accepting new connections, it seems I need to write that active thread list.

StopListening() will stop accepting new clients without shutting down the server completely.  But you can't transfer the existing clients to another server, if that is your goal.

Edited February 14 by Remy Lebeau

[mjustin 26]

1 hour ago, narag said:

That's not very nice to the clients. My intention is to hot swap servers gracefully, without causing connections errors. Even StopListening seems to close threads' sockets.

HTTP is a stateless protocol. Therfore, closing the connection should not cause any client-side errors. And connections are not "transferable" between processes or machines.

 

Edited February 14 by mjustin

[Remy Lebeau 1498]

Just now, mjustin said:

HTTP is a stateless protocol. Therfore, closing the connection should not cause any client-side errors.

Unless the connection is closed while the server is in the middle of exchanging an HTTP message, that is. Then the client will likely get a transmission error, and have to retry the request that aborted. 

[narag 0]

34 minutes ago, Remy Lebeau said:

Meaning what, exactly? What are you actually trying to accomplish?

Updating the server.

 

The traffic is diverted to a new instance with updated code, then the "old" server is shut down, but if there are clients already connected to the instance that is shutting down, the transactions must be completed, not interrupted. There are a few endpoints that carry lengthy database operations, not so long as to cause timeouts, but long enough, half a minute sometimes, to give them a little leeway.

 

Of course I didn't mean migrating the transactions to another process, that would be... an interesting problem 

 

44 minutes ago, Remy Lebeau said:

StopListening() merely closes the listening sockets and their threads. It does not close any clients or stop their threads. Closing a listening socket does not close any clients that connected to it, as those are independent sockets once they have been accepted.

 

So Active is safe... that's great, so I can clean the connection pool just sequentially after Active := False.

 

Thank you so much!

[Remy Lebeau 1498]

56 minutes ago, narag said:

Updating the server.

The traffic is diverted to a new instance with updated code, then the "old" server is shut down

How are you "updating code" for a running instance?

56 minutes ago, narag said:

if there are clients already connected to the instance that is shutting down, the transactions must be completed, not interrupted.

You can StopListening() to let existing clients finish their work, but then you will have to manually monitor the client threads to detect when they have all terminated.  Such as by monitoring the TIdHTTPServer.Contexts.Count until it falls to 0.

 

Also, if you have KeepAlives enabled on the server, a client thread will not terminate until the client disconnects after their own KeepAlive timeout has elapsed.  TIdHTTPServer does not implement KeepAlive timeouts on its end yet (work in progress), but you can use a ReadTimeout to fake it.

 

Also, with KeepAlives enabled, clients could continue to send new requests as long as the KeepAlive doesn't elapse.  If you want to prevent further requests, you would have to set a flag during shutdown and then check that flag in your OnCommand... events so you can close connections gracefully (ie, AResponseInfo.CloseConnection = True).

[narag 0]

2 hours ago, Remy Lebeau said:

How are you "updating code" for a running instance?

I write the new .exe with updated code to another location in the the disk, I execute it, listening in a new port and I tell the load balancer to send all new requests to that port. Then I do whatever I need to do ("TIdHTTPServer.Active := False" or TIdHTTPServer.StopListening" or custom code if it's not enough) to tell the old instace to stop accepting connections and then process the standing ones before shutdown.

 

In other words: it's the way to update a working service without clients even noticing it. The updates would be for non-critical errors or enhancements, so it makes sense to keep the old code working until it can be replaced.

 

2 hours ago, Remy Lebeau said:

You can StopListening() to let existing clients finish their work, but then you will have to manually monitor the client threads to detect when they have all terminated.  Such as by monitoring the TIdHTTPServer.Contexts.Count until it falls to 0.

I had understood that "Active := False" would call StopListening so the pending transactions would complete before the instruction following "Active := False" is executed 🤔

 

Should I call StopListening on its own instead of "Active := False" instead?

 

2 hours ago, Remy Lebeau said:

Also, if you have KeepAlives enabled on the server, a client thread will not terminate until the client disconnects after their own KeepAlive timeout has elapsed.  TIdHTTPServer does not implement KeepAlive timeouts on its end yet (work in progress), but you can use a ReadTimeout to fake it.

 

No, it's a REST web service, only used by other programs, not browsers. I'll make sure that KeepAlives is disabled, thank you for the heads up.

[Remy Lebeau 1498]

3 hours ago, narag said:

I write the new .exe with updated code to another location in the the disk, I execute it, listening in a new port and I tell the load balancer to send all new requests to that port. Then I do whatever I need to do ("TIdHTTPServer.Active := False" or TIdHTTPServer.StopListening" or custom code if it's not enough) to tell the old instace to stop accepting connections and then process the standing ones before shutdown.

Oh, OK.

Quote

I had understood that "Active := False" would call StopListening so the pending transactions would complete before the instruction following "Active := False" is executed 🤔

Setting Active=False does stop listening for new connections, but it ALSO immediately closes any existing client connections, too.  Then it waits for everything to finish cleaning up before it exits back to your code.

Quote

Should I call StopListening on its own instead of "Active := False" instead?

If you want existing clients to continue their work uninterrupted, then yes.  Try something like this:

ShuttingDown := True;
IdHTTPServer1.StopListening;
Timer1.Enabed := True;
...
procedure TMyForm.Timer1Timer(Sender: TObject);
begin
  with IdHTTPServer1.Contexts.LockList do
  try
    if Count > 0 then Exit;
  finally
    IdHTTPServer1.Contexts.UnlockList;
  end;
  Timer1.Enabled := False;
  IdHTTPServer1.Active := False;
  Application.Terminate;
end;
...
procedure TMyForm.IdHTTPServer1CommandGet(AContext: TIdContext;
  ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo);
begin
  if ShuttingDown then
  begin
    AResponseInfo.ResponseNo := 503;
    AResponse.CustomHeaders.Values['Retry-After'] := '5';
    AResponseInfo.CloseConnection := True;
    Exit;
  end; 

  // process normally ...
  if ShuttingDown then
    AResponseInfo.CloseConnection := True;
end;

 

Edited February 14 by Remy Lebeau

[narag 0]

On 2/15/2025 at 12:58 AM, Remy Lebeau said:

If you want existing clients to continue their work uninterrupted, then yes.  Try something like this:

Thank you, that seems to cover all the needs