UDP sending and receiving

[TurboMagic 92]

I'm using D10.4.2 and current ICS from GetIt.

I'd like to do the following thing but struggle and the demos I have found so far show only one part of the thing each but not how to combine both.

 

I want to send something to an UDP device with unknown address, so I need to send as broadcast. Port is know. Any device listening on that port
will send something defined back to me which I want to receive. Sending part already works but OverbyteIcsUdpSend1 example only shows how to send and

OverbyteIcsUdpLstn only shows how to listen. Calling Listen anywhere in my code raises an exception

 

... an Exception of class ESocketException with message 'Address not available (#10049 in listen: Bind)' occurred.

 

  FWuTSocket := TWSocket.Create(nil);

  FWuTSocket.SocketFamily       := sfIPv4;
  FWuTSocket.Addr               := '255.255.255.255'; 
  FWuTSocket.MultiCast          := FALSE;
  FWuTSocket.MultiCastAddrStr   := '';
  FWuTSocket.Proto              := 'udp';
  FWuTSocket.Port               := 1234;
  FWuTSocket.LocalAddr          := '0.0.0.0';
  FWuTSocket.LocalPort          := '0';
  FWuTSocket.LineMode           := false;
  FWuTSocket.OnSessionConnected := OnWuTSearchSessionConnected;
  FWuTSocket.OnDataAvailable    := OnWuTSearchDataAvailable;
  FWuTSocket.OnSessionClosed    := OnWuTSearchSessionClosed;
  FWuTSocket.Connect;

Where to put listen? Or do I need a 2nd socket? If yes, how to set that up so it actually receives the answer(s) of the device(s)?
Set the other one in the OnWuTSearchSessionConnected event up and use FWuTSocket.LocalPort as local port?

 

Since I do not have the right device at hand right now I modified OverbyteIcsUdpLstn example to send something back.
When that one receives a message it does this:

 

procedure TMainForm.WSocketDataAvailable(Sender: TObject; Error: Word);
var
    Buffer : array [0..1023] of AnsiChar;
    Len    : Integer;
    Src    : TSockAddrIn6;
    SrcLen : Integer;
begin
    if FSenderAddr.sin6_family = AF_INET then
    begin
        SrcLen := SizeOf(TSockAddrIn);
        Len    := WSocket.ReceiveFrom(@Buffer, SizeOf(Buffer), PSockAddr(@Src)^, SrcLen);
        if Len >= 0 then
        begin
            if (PSockAddr(@FSenderAddr).sin_addr.S_addr = INADDR_ANY) or
               (PSockAddr(@FSenderAddr).sin_addr.S_addr = PSockAddr(@Src).Sin_addr.S_addr) then begin
                Buffer[Len] := #0;
                DataAvailableLabel.Caption := IntToStr(atoi(DataAvailableLabel.caption) + 1) +
                                          '  ' + String(WSocket_inet_ntoa(PSockAddr(@Src).sin_addr)) +
                                          ':'  + IntToStr(WSocket_ntohs(PSockAddr(@Src).sin_port)) +
                                          '--> ' + String(Buffer);
            end;
            
          WSocket.Close;
          WSocket.SocketFamily := sfIPv4;
          WSocket.Addr         := String(WSocket_inet_ntoa(PSockAddr(@Src).sin_addr));

          WSocket.Port       := IntToStr(WSocket_ntohs(PSockAddr(@Src).sin_port));
          WSocket.OnSessionConnected := WSocketSessionConnectedAfterReceive;
          WSocket.Connect;

And in the OnSessionConnected event is sends the contents of a packed record like this:

WSocket.Send(@FWuT, SizeOf(FWuT));

 

It is sure that it reads the sending code, but on the other end (that first application which sends a message to the modified OverbyteIcsUdpLstn example) nothing is ever received.

[Angus Robertson 574]

UDP is connectionless protocol, connect does nothing, you use SendTo and SendTo6 to send UDP packets setting the address and port in a TSockAddrIn or TSockAddrIn6 structure.  When receiving, in onDataAvailable you use ReceiveFrom or ReceiveFrom6 which fills the same structure with the remote address, conveniently so you can reply.

 

Angus

 

[TurboMagic 92]

Thanks so far, but I still don't properly manage this.
 

In my application I try to send like this, but in the UDP listener demo nothing arrives and SentTo returns -1:

 

procedure TDeviceSearch.OnWuTSearchSessionConnected(Sender: TObject; Error: Word);
var
  Src    : TSockAddrIn;
  SrcLen : Integer;
  Data   : RawByteString;
begin
  if (Error = 0) then
  begin    
    try
      SrcLen := SizeOf(TSockAddrIn);
      // I want to send to everybody I can reach (broadcast)
      Src.sin_addr.S_addr := WSocket_inet_addr('255.255.255.255');
      Src.sin_port := 1234;
      Src.sa_family := AF_INET;

      Data := 'Hallo?';
      // This posts -1 into my log. I'd expect 6 instead...
      log.Send('Bytes: ' + (Sender As TWSocket).SendTo(Src, SrcLen, @Data[low(Data)], Length(Data)).ToString);
    except
      On E:Exception do
        log.SendException'Failure in OnWuTSearchSessionConnected: ' + E.Message, E);
    end;
  end
  else
    log.Send('Failure in UDP handling: ' + Error.ToString);
end;

What am I making wrong/missing?

Edited June 1, 2021 by TurboMagic

[Angus Robertson 574]

Your partial code looks okay. TCP Send and related functions simply stuff data in a buffer with no real error, SendTo sends UDP immediately with a Windows API but unfortunately does not get the real error (which Send already does for TCP). 

 

If you get back -1 from SendTo, use LastError := WSocket_Synchronized_WSAGetLastError;  then WSocketErrorDesc to get a message, I'll correct this in SVN for V8.67, not sure how it got missed for 20 years, but UDP rarely fails, or at least rarely gives errors, you have no idea if packets are received anywhere.

 

Angus

 

 

 

 

 

[Angus Robertson 574]

Further to my original message about using SendTo, for broadcasting or multicasting you must also set property Addr to the broadcast address before calling Connect, otherwise the socket will not be opened with the correct SO_BROADCAST flag.   Send should also work, as used in various components and samples, but SendTo is required for UDP servers to reply to the source address, and works as well for clients.

 

Angus

 

[TurboMagic 92]

Hello,

 

I added your error message code but I had to modify it.

I had to use this one: WSocket_WSAGetLastError because the other one is not exported in OverbyteIcsWSocket.pas.

 

I get the following error message now: "Permission denied" ErrorCode: 10013

 

TurboMagic

[TurboMagic 92]

From this albeit German description here:

https://helpdesk.ebertlang.com/kb/a507/winsock-fehlercode-beschreibung.aspx

Ok, found the exact same description in English on a Microsoft site:

https://docs.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2
 

I take it that I should call this one:

setsockopt(SO_BROADCAST)

 

because I'm using SendTo in conjunction with a broadcast address.

Any idea how I'd call this via ICS and if this is really a good idea?
 

The other idea would be to use a variant of my old code which fetched a list of all network interfaces and looped through those
and send the request on each one separately.

 

TurboMagic

Edited June 1, 2021 by TurboMagic

[Angus Robertson 574]

II think our messages overlapped.

 

Angus

 

 

 

[TurboMagic 92]

Another small question: I had seen that TWSocket offers a LastError property as well.
But when replacing WSocket_WSAGetLastError  with it the error code is 0.

What is this meant for?

[TurboMagic 92]

30 minutes ago, Angus Robertson said:

Further to my original message about using SendTo, for broadcasting or multicasting you must also set property Addr to the broadcast address before calling Connect, otherwise the socket will not be opened with the correct SO_BROADCAST flag.   Send should also work, as used in various components and samples, but SendTo is required for UDP servers to reply to the source address, and works as well for clients.

 

Angus

 

Hm ok, but it looks like I don't yet understand 100% what I need to do to make it work.

This is my code for setting up the socket:

 

  FWuTSocket.SocketFamily       := sfIPv4;
  FWuTSocket.Addr               := '255.255.255.255'; //ICS_ANY_HOST_V4; //'255.255.255.255';
  FWuTSocket.MultiCast          := FALSE;
  FWuTSocket.MultiCastAddrStr   := '';
  FWuTSocket.Proto              := 'udp';
  FWuTSocket.Port               := '0'; //cWuTUDPPort;
  FWuTSocket.LocalAddr          := '0.0.0.0';
  FWuTSocket.LocalPort          := '0';
  FWuTSocket.LineMode           := false;
  FWuTSocket.OnSessionConnected := OnWuTSearchSessionConnected;
  FWuTSocket.OnDataAvailable    := OnWuTSearchDataAvailable;
  FWuTSocket.OnSessionClosed    := OnWuTSearchSessionClosed;
//  FWuTSocket.Connect;
  FWuTSocket.Listen;

I thought I need to use listen instead of connect so I can receive data on that socket as well. But when I change Addr to the above 255.255.255.255 I get an

"address not available" #10049  in listen: Bind error from Winsock...

 

Now how to set up the socket properly to send via broadcast but listen for the answers as well?

[Angus Robertson 574]

LastError is generally set when a Windows API fails to that Windows error, but code is needed after every such function call, and was sometimes missing, for SendTo/6 in particular, now fixed. 

 

You must use Connect for broadcast to work.  You should use a separate socket for listening. 

 

Angus

 

[Remy Lebeau 1403]

On 5/31/2021 at 5:54 AM, Angus Robertson said:

UDP is connectionless protocol, connect does nothing

Connect()'ing a UDP socket assigns the specified peer IP/port as a static association, so that subsequent sends always go only to that peer, and only packets from that peer can be received. It also enabled better error reporting.  If a send fails with an ICMP error, such as host/port unreachable, a subsequent send/read can fail with an appropriate error code.

On 5/31/2021 at 5:54 AM, Angus Robertson said:

you use SendTo and SendTo6 to send UDP packets ... When receiving, in onDataAvailable you use ReceiveFrom or ReceiveFrom6

On a Connect'ed socket, you can use Send() and Receive(), you are not restricted to SendTo() and ReceiveFrom().

[Remy Lebeau 1403]

3 hours ago, TurboMagic said:

I thought I need to use listen instead of connect so I can receive data on that socket as well.

You don't need to Listen() on a UDP socket.  Only Bind() it, and then ReceiveFrom() on it.  Listen() is only used for TCP.

Quote

But when I change Addr to the above 255.255.255.255 I get an

"address not available" #10049  in listen: Bind error from Winsock...

You can't Bind() to a broadcast IP, only to a local IP.

Edited June 1, 2021 by Remy Lebeau

[TurboMagic 92]

If I should use a 2nd socket for listening, how would I set that up? Do a connect with the sending one first, so it can send as broadcast and in the OnSessionConnected create the 2nd socket so that it can listen on the port the sending one used?
The party receiving and replying to my broadcast needs to send the answer to the port from which the broadcast message was sent, otherwise it wouldn't know where to send it to (and I don't have influence on how the other device does this anyway).

[Angus Robertson 574]

For your broadcast socket, see what replies you receive in the onDataAvailable event, if any.  Test it.  

 

The OverbyteIcsIpStmLogTst.dpr sample allows all this to be tested using the TIcsIpStrmLog component that does UDP client and server, only snag is a bug in the sample that meant UDP server alone did not listen correctly, fixed yesterday when I tested broadcasting, I'll put it in SVN shortly. 

 

Angus

 

[TurboMagic 92]

Something else: when trying to open the Delphi\AllDemos project group (current ICS version from GetIt) I get some failures.
One is: Failure reading SslSmtpTestForm -> Class TIcsRestEMail not found. The same class is not found for POP3ExcercizerForm and HttpRestForm.

For IcsIsapiWebModule failures for properties do not exist  are shown for the following properties: ClientHeight, ClientWidth, Color, Font.Charset,
Font.Color, Font.Height, Font.Style, PixelsPerInch, TextHeight.

Are they known to you?

[Angus Robertson 574]

And how is this relevant to your UDP problems, is it working yet. 

 

Angus

 

[TurboMagic 92]

This is not relevant to my still not working UDP problem. I just tried to find that demo quicker by opening that project group and thought I report the errors popping up loading that group so they can be fixed eventually.

 

Back to my problem:

The demo you mentioned is a quite big and complex one and the component used in it is equally big.

So I tried to go back to the simple pair of OverbyteIcsUdpSend and OverbyteIcsUdpLstn demos.

Btw. at lease those two demos are set to release mode and thus have no debug info and breakpoints are disabled though.

In my view it would make more sense to have debug mode and debug info on in the demos.

The listen one does get a packet on the port it listens and it sends something back to the sender's port.
I write that in a memo for logging now and the port number logged as LocalPort on the sending site is
the same as on the receiving side.

 

Now the only thing left is to receive something on the UDP sending socket. DataAvailable event is never called and
if I tried to set up a 2nd socket as recommended by you that fails on Listen call because the port assigned, the LocalPort
of the sending socket is of course already in use. But that is the port the sender will get the reply to his request, as that
is the port number the receiver gets by receiving the request.

 

I could try Remy's ReceiveFrom but that seems to be a synchronized/blocking call, which is of course not desired.

 

So I'm a bit puzzled and think there must be some easier solution.

 

Strange thing is, that the old code I'm trying to replace simply defined Addr as 255.255.255.255, set Port, set local address
to the IP of the interface it was sending out the broadcast for (it looped through all interfaces on that PC), did a Connect
on that socket and then received an answer in OnDataAvailable event. It has to be replaced because after sending I entered

a loop calling Socket.MessagePump; for a certain time, which is of course no good and often crashes.

 

Now the only thing in my new code is, that OnDataAvailable on the sending socket is never called but the receiver does a

WSocket.SendTo(PSockAddr(@Src)^, SrcLen, @FWuT, SizeOf(FWuT)); in OnDataAvailable. And since I added this to your example
Src is populated by ReceiveFrom and thus contains the correct data.

 

 

Hey, I just have found some solution now, but I don't think this is perfect.

I just saw, that you are closing the sending socket in the sender demo in OnDataSent event.

When I set up a 2nd socket directly after connect of the first I get the LocalPort but I don't
do Listen yet.

 

Now in OnDataSent I do close the sending socket and then do a Listen on the other one and
in my example that one now receives OnDataAvailable and thus the data sent back by the receiver demo.

[TurboMagic 92]

I just found a little bit of time now and made a new attempt.
I record what's happening using Wireshark now and that clearly shows that my device receives my message and it sends something back to the local port my socket got assigned by Windows.

But my application never receives anything.

 

I used the UDPSend example as starting point. I initially had tried out this 2 sockets solution but since it didn't work I changed it. I added a new socket and dont use the other one at the moment.

Here's the relevant code part:

 

procedure TMainForm.SendButtonClick(Sender: TObject);
begin
  WSocketReceive.Proto      := 'udp';
  WSocketReceive.SocketFamily := sfIPv4;
  WSocketReceive.Port      := '0';
  WSocketReceive.Addr      := '0.0.0.0';
  WSocketReceive.Listen;

  MemoReceive.Lines.Add('Local Port: ' + WSocket.LocalPort);
end;

procedure TMainForm.WSocketReceiveDataAvailable(Sender: TObject; ErrCode: Word);
begin
 // This is never called!!!
 MemoReceive.Lines.Add(TimeToStr(Now) + ' ' + WSocketReceive.ReceiveStr);
end;

procedure TMainForm.WSocketReceiveSessionConnected(Sender: TObject;
  ErrCode: Word);
var
  Src : TSockAddr;
  Str : RawByteString;
begin
  Src.sin_family := SOCK_DGRAM;
  Src.sin_port   := 8513;
  // I don't know how to write this one less cumbersome:
  Src.sin_addr.S_un_b.s_b1 := #255;
  Src.sin_addr.S_un_b.s_b1 := #255;
  Src.sin_addr.S_un_b.s_b1 := #255;
  Src.sin_addr.S_un_b.s_b1 := #255;

  // This gets sent:
  WSocketReceive.SendTo(Src, SizeOf(Src), BytesOf(Str), Length(Str));
end;

procedure TMainForm.WSocketSessionAvailable(Sender: TObject; ErrCode: Word);
begin
  // This is never called
  MemoReceive.Lines.Add('Port: ' + WSocketReceive.Port);
end;

As you can see I added a memo for output. But the only thing ever shown in it is the 'Local port' message posted directly after Listen.

 

What am I'm doing wrong?

Edited June 10, 2021 by TurboMagic
Fixed typo in comment

[FPiette 384]

WSocketReceive.Port      := '0';

This is clearly wrong. Use a valid port number.

 

procedure TMainForm.WSocketSessionAvailable(Sender: TObject; ErrCode: Word);
begin
  // This is never called
  MemoReceive.Lines.Add('Port: ' + WSocketReceive.Port);
end;

UDP is a sessionless protocol.