Delphi 10.3.3 - Indy from 0ba2688b Git "Hello World"

[aehimself 399]

Hello,

 

I started to experiment with Indy (looking for a worthy successor to port some legacy TServerSocket-TClientSocket apps to). As Emba had some issues with Indy lately I decided to wipe it off, pulling an up-to-date version from https://github.com/IndySockets/Indy/ and installing that one.

I have a deadlock issue with my hello world app, where it seems that TidIOHandler.ReadFromSource does not return and my project throws a "disconnected" exception upon exiting. Setup is fairly simple: TidTCPServer with defaultport set to 1024, TidTCPClient with host set to 127.0.0.1 and port set to 1024.

 

The whole project contains the following:

 

procedure TForm1.Button1Click(Sender: TObject);
begin
 IdTCPServer1.Active := True;
 IdTCPClient1.Connect;
 IdTCPClient1.IOHandler.WriteLn('Hello, world');
end;

procedure TForm1.IdTCPServer1Execute(AContext: TIdContext);
Var
 s: String;
begin
 s := AContext.Binding.PeerIP + ': ' + AContext.Connection.IOHandler.ReadLn;
 TThread.Queue(nil, procedure Begin Memo1.Lines.Add(s) End);
end;

TCPServerExecute get hits twice. First, it reads "Hello, world" and puts it to the memo correctly. The second time, it goes into the method mentioned above and never returns (can not debug the exact location, as my Delphi is showing the line indicator somewhere random...).

 

Again, this is my first test run of Indy (copied the code from https://stackoverflow.com/questions/31039580/how-can-i-send-and-recieve-strings-from-tidtcpclient-and-tidtcpserver-and-to-cr) and have zero knowledge in the component yet. Am I doing something wrong? What would be the smallest, least complicated solution for me so I can start my learning curve?

 

Thanks!

 

[Kryvich 165]

No problem here with Indy components included in Delphi 10.3.3 package. Just change Button1Click as follows:

  if not IdTCPServer1.Active then begin
    IdTCPServer1.Active := True;
    IdTCPClient1.Connect;
  end;
  IdTCPClient1.IOHandler.WriteLn('Hello, world');

I did not find projects IndyCore260, IndyProtocols260 etc. for the new Delphi 10.3.3 in the library on GitHub.

[aehimself 399]

@Kryvich Thank you for the suggestion, but that will only get rid of an exception "Already connected" if the button is pressed twice during the lifetime of the application. I know it's not a nice and error handling code, it was meant as a test run only. Emba broke Indy's SSL certificate handling somehow (see: https://community.idera.com/developer-tools/b/blog/posts/rad-studio-10-3-3-indy-server-ssl-certificate-patch) and despite having a patch I feel more comfortable using a more up-to-date source.

There are no 260, only 250 packages, but they compile and install nicely on 10.3.3.

 

Does the deadlock appear if you run the code above? I had the same experience with the (unpatched) Emba version and the latest Git snapshot version too.

[Kryvich 165]

No, there is no deadlock here.

[aehimself 399]

While the program is running, it works for me too, I only get an error message when I close it:

 

 

 

The strange thing is that ServerExecute gets hit twice only at the first button press, consecutive WriteLn-s trigger the event only once.

 

And let me correct myself - it's not a deadlock, simply a blocking read operation from a socket, which has nothing in the queue. Me thinks, that is.

[Kryvich 165]

If I put a breakpoint to the line TThread.Queue(...) it gets hit twice on every button press. I think it's how the debugger works. I've added a counter and there is no any missed clicks. And no error messages on exit.

procedure TForm1.Button1Click(Sender: TObject);
begin
  if not IdTCPServer1.Active then begin
    IdTCPServer1.Active := True;
    IdTCPClient1.Connect;
  end;
  Inc(ClickCount);
  IdTCPClient1.IOHandler.WriteLn(Format('Hello, world. Click #%d.',
    [ClickCount]));
end;

procedure TForm1.FormCloseQuery(Sender: TObject; var CanClose: Boolean);
begin
  if IdTCPServer1.Active then
    IdTCPClient1.Disconnect;
end;

procedure TForm1.IdTCPServer1Execute(AContext: TIdContext);
var
  s: string;
begin
  s := AContext.Binding.PeerIP + ': ' + AContext.Connection.IOHandler.ReadLn;
  TThread.Queue(nil, procedure begin Memo1.Lines.Add(s) end);
end;

 

 

[Kryvich 165]

I think I know why you get the exception. Put this code to the FormCloseQuery:

  if IdTCPServer1.Active then begin
    IdTCPServer1.Active := False;
    IdTCPClient1.Disconnect;
  end;

You'll get the exception. Then swap the lines:

    IdTCPClient1.Disconnect;
    IdTCPServer1.Active := False;

No exception.

[aehimself 399]

Actually, closing the client socket before exiting does get rid of the exception!

I made some further tests, and indeed it seems that it is a debugger issue. Put a breakpoint on s := and press F8. First run it does, second time it does not reach the TThread.Queue line. Now put a breakpoint on both; F8 will step through them nicely.

I also think I get the point of the double Execute event... first time is does not read anything from the socket - I guess the context is just being initialized. On the second run text gets read successfully.

 

So it seems the Hello World implementation is working properly, I just hit an other debugger anomaly 😞

 

Thank you very much for your help! 🙂

[Remy Lebeau 1436]

21 hours ago, aehimself said:

TCPServerExecute get hits twice. First, it reads "Hello, world" and puts it to the memo correctly. The second time, it goes into the method mentioned above and never returns (can not debug the exact location, as my Delphi is showing the line indicator somewhere random...).

This is normal behavior!

 

Your client is sending only 1 line of text.  The server's OnExecute event is looped, and so it tries to read 2 lines.  There is no data for the 2nd read is receive, so it blocks the calling thread waiting for new data that will never arrive, until the client disconnects or the server is deactivated.  At which time, the blocked read will then fail and raise a socket exception.  Let it happen that way!  The server will handle the exception for you and stop the thread that called the OnExecute event.  You will see the exception only if you are running your code in the IDE's debugger (and have not configured the debugger to ignore the exception), or have assigned a handler to the server's OnException event.

 

This is how Indy is designed to work!

 

To help illustrate this better, you should move your client's IOHandler.Write() call into a button OnClick event, and then press it multiple times while the TIdTCPClient is connected to the server, like Kryvich described (though I would have used separate buttons for Connect, Disconnect, and Write()).  You will see the ReadLn() call unblock with each button click.

Quote

Again, this is my first test run of Indy (copied the code from https://stackoverflow.com/questions/31039580/how-can-i-send-and-recieve-strings-from-tidtcpclient-and-tidtcpserver-and-to-cr) and have zero knowledge in the component yet.

Then you should read Introduction to Indy.

Quote

Am I doing something wrong?

No.

Quote

What would be the smallest, least complicated solution for me so I can start my learning curve?

You already have it.  It doesn't get much simpler than what you already have.

Edited January 31, 2020 by Remy Lebeau

[Remy Lebeau 1436]

11 hours ago, Kryvich said:

I did not find projects IndyCore260, IndyProtocols260 etc. for the new Delphi 10.3.3 in the library on GitHub.

That is because they have not been created and checked in yet.

[Remy Lebeau 1436]

8 hours ago, aehimself said:

Actually, closing the client socket before exiting does get rid of the exception!

Closing the client socket while the server is active will still raise an exception on the server side, such as EIdConnClosedGracefully.  The debugger can be configured to ignore that exception (IIRC, it already is by default).

 

Deactivating the server while the client is still connected will close the socket connected to the client, and thus will usually also raise an exception on the server side if the OnExecute event is still blocked on a read/write operation.

8 hours ago, aehimself said:

I also think I get the point of the double Execute event... first time is does not read anything from the socket - I guess the context is just being initialized. On the second run text gets read successfully.

You have it backwards.  The first triggering of the OnExecute will successfully read your initial text from the socket, the second triggering will not until you send more text.

[aehimself 399]

Good day Remy, I was wondering when you'll show up 🙂

 

So it seems the way was right from the beginning, I just had no idea Indy is automatically creating a thread for each connection and simply hang them while waiting for data. Sounds logical.

19 minutes ago, Remy Lebeau said:

Then you should read Introduction to Indy.

To be completely honest I never had much luck with documentations. I prefer the bang-my-head-against-a-brick-wall method, which is a slower learning curve but makes the knowledge to stick around.

I smiled on this though:

    // Sleep is substituted for a long DB or other call
    Sleep(5000);

Some of our DB actions at work take 30+ seconds, especially on a shared test DB or a production database over the Internet.

 

Well, with the very basics covered I can slowly can move to more advanced features (DoS protection, throttling, timeouts and so on) and if I like Indy I might even change the webserver component I'm using to create a REST-like interface.

Just one question though. Are all the events of the component are called from within the connections thread context? Where should I prepare locks / synchronizations?

[Remy Lebeau 1436]

2 hours ago, aehimself said:

I just had no idea Indy is automatically creating a thread for each connection and simply hang them while waiting for data.

Yes, that is what it does.

Quote

Are all the events of the component are called from within the connections thread context?

The OnConnect, OnDisconnect, OnExecute, and OnException events are called in the context of the connection thread, yes.

Quote

Where should I prepare locks / synchronizations?

That is a bit broad to answer.  It really depends on how and where they are being used.  I'm assuming you are referring to performing per-thread preparations, right?  Then it comes down to whether you are using Indy's default 1-thread-per-connection model, or using thread pooling instead.

 

If the former, then the server's OnConnect and OnDisconnect events will usually suffice.  If you do not assign any component to the server's Scheduler property, a TIdSchedulerOfThreadDefault component is created and assigned for you automatically when you activate the server.  TIdSchedulerOfThreadDefault creates a new thread per connection, and frees each thread when the connections are closed.  Thus the server's OnConnect and OnDisconnect events are called only once per thread.

 

If the latter, then as connections are closed and threads get reused, the OnConnect and OnDisconnect events can be called for multiple connections on the same thread, so that wouldn't really be a good place to perform per-thread initializations/cleanups (per-connection, yes).  A better option (at least for now) is to derive a new class from TIdThreadWithTask and have it override the virtual BeforeExecute() and AfterExecute() methods to perform per-thread initializations/cleanups as needed, and then you can explicitly assign a TIdSchedulerOfThreadPool component to the server's Scheduler property, and assign your Task class to the TIdSchedulerOfThreadPool.ThreadClass property.

 

Technically, you could take the same Task approach with TIdSchedulerOfThreadDefault, too.  But that would generally be overkill, but usable.

Edited January 31, 2020 by Remy Lebeau