Jumbo packet use with ICS

[mmcgaw 0]

I have used ICS for a very long time, and while building a new app, I found the need to go to jumbo packets.

 

The app does UDP datagrams, sending and receiving.  I have two Win10 machines with their ethernet adapters set to 4K MTU frames, connected by a single CAT6A cable.  No routers, no switches.  We run direct, machine to machine.

 

I am able to ping each machine from the other.  I am able to test, via ping, the ability to send and receive 4K MTU frame sizes.

 

All good, here.

 

The problem comes when I try to send a UDP packet with a size of 3628 bytes.  This should be well short of the 4K frame size.  What I see, is three successive packets received on the other end.  The first packet is decoded properly (had a message byte and it correctly interpreted), the next two packets come immediately after the first, and do not have a legitimate message byte.  

 

Here's the weird thing:  If I grab the length of each of the three packets, guess what:  they add up to 3628 (!).  Somehow, it seems, the sending process is dividing the original packet into 3 smaller packets, the first two at the maximum frame size (here, 1460), and the third packet is 708 bytes.

 

Now, in the interest of full disclosure:  I run D7, and I _had_ been running a version of ICS that dates to 2006 or so.    This is the platform that surfaced the issue.

 

I updated my ICS to the latest on Github (8.64, I believe), and got my app rebuilt with this updated version (again, on D7).  I am still getting the mysterious receipt of three packets, initiated by the sending of one jumbo packet.  And collectively, each volley of three sum to the size of the single packet that was sent.

 

Is there anything special I need to know to take advantage of the jumbo packet size in ICS?  I property setting somewhere that I have missed, perhaps?

 

Thanks!

 

M

[SwiftExpat 65]

Not related to ICS, but sounds like a problem I had in the past. Disable flowcontrol on the ethernet adapter.

 

 

[Angus Robertson 527]

The ICS github repository is unofficial and never updated, it should be removed to avoid confusion with https://svn.overbyte.be/svn/ where you will find the latest ICS.

 

Never tested ICS with jumbo packets, there are no special settings I'm aware of.  More likely to be an OS issue, what are you using?

 

Angus

[FPiette 367]

Did you try setting SocketSndBufSize, SocketRcvBufSize and BufSize? The first two are passed to Winsock to set Windows own buffer size. The later (BufSize) is used by TWSocket to manage his linked list of buffers for synchronous send.

[mmcgaw 0]

Thank you one and all for your kind thoughts and suggestions on this.  I will look into the flowcontrol setting for the ethernet cards, and will look at setting the BufSize parameters (Snd/Rcv/BufSize) to see which of these may solve the matter.

 

Also, in response to another's inquiry as to the OS being used:  Win10 pro on one machine, Win10 Home on the second machine. 

 

I will report back on the results-

 

Again, much appreciated!

 

-M

[mmcgaw 0]

OK, I have done a bit of investigating on the suggestions made.  SwiftExpat suggested looking at flowcontrol for the ethernet adapters.  Both adapters had been set to auto RX/TX, and I disabled this on both.  No change.

 

With regard to Mr. Piette's suggestion on (Set)BufSize, (Set)SocketRcvBufSize and (Set)SocketSndBufSIze:  These items do not show up on the component property panel, and investigating the WSocket.pas code for these items, I see that in 2005, the BufSize was set to 1460 and the SocketSndBufSize was set to 8192.  This was most instructive, as I see my incoming packet (size of 3628) get received as 3 packets: 2 of size 1460 and the last one of size 708 (sums to 3628).  So the BufSize appears to be involved somehow, somewhere.  As ICS evolved, more and more to do with BufSize, SocketSndBufSizeSndBufSize and SocketRcvBufSize was done.  In particular, 'setters' were written for these.  However, it seems that these setter procs are not available to the application programmer for the standard component?  At least code completion does not see them.

 

Can someone suggest a short code sequence where these can be called to positive effect?

 

I think the change to the buffer should be done in the code sequence that causes the socket to Listen:

Maybe like this:

 

    WSocket.Proto             := 'udp';
    WSocket.Addr              := '0.0.0.0';
    WSocket.Port              := '9870';

    WSocket.SetRcvBufSize(3840);  //15*256, > needed rcv packet size coming in
    WSocket.Listen;

 

I guess the same thing has to be done for the send socket, to ensure it's packet gets sent as one block.

 

Is this how it's done, then?

 

Thanks!

 

M

 

[FPiette 367]

BufSize is only important for sending. For UDP, it must be large enough to contain the largest packet to be sent. Buf size is an internal TWSocket working thing.

SocketRcvBufSize and SocketSndBufSize is directly passed to winsock API and it is quite a complex topic. See Microsoft documentation at https://learn.microsoft.com/en-us/troubleshoot/windows/win32/data-segment-tcp-winsock

 

Also note that the network layer may break packets into smaller one. You must set jumbo packet on all devices between sender and receiver.

 

Use WireShark or other protocol analyzer to see if the packet fragmentation is done at sending side (You see it in the protocol analyzer) or into winsock at receiver side (You don't see it in protocol analyzer).

 

[mmcgaw 0]

Hello Mr. Piette:

 

I found how to set BufSize.  I did not see it in the component viewer, but code complete did indeed show it as a property that can be set.  My packet size is 3628, so I set BufSize to 4000.  That worked!  THANK YOU!  I had already set both ethernet 'boards' to jumbo packets at the 4K MTU size many days ago, so that was in good shape-  I knew it worked, as running ping with the -f and -l options and a packet size, demonstrated the packets were having no trouble with loss or fragmentation, at least at the ping level of transmission performance.

 

What I did find, was this:  I also sought to initially set SocketSndBufSize and SocketRcvBufSize to at least 4000, but I got run time errors when I started the socket system (e.g., placing a socket for receiving in Listen mode)  This raised an issue with an option that was required: 'SO_RCVBUF'.  The send side has a similarly named option that somehow must be transmitted to the Socket component or subsystem.   You alluded to this being a complex matter, and cited a Microsoft reference.  Are these two settings (SocketSndBufSize and SocketRcvBufSize) not to be altered?  If one can alter the sizes, how does one deal with including SO_RCVBUF and/or SO_SNDBUF in the socket options?

 

I will pick up Wireshark should this be necessary- right now, things are working well, and I am heading toward endurance/performance testing next, as I must send these packets at a rate of 30 per second, possibly much more, so I will see how this all performs.   Fragmentation due to a router or switch cannot occur for this app because the machines are directly connected.

 

Thank you very much for your kind attention and patience!

 

-M

 

In any case, simply moving the BufSize up solved the issue.

[FPiette 367]

11 hours ago, mmcgaw said:

That worked!  THANK YOU! 

In any case, simply moving the BufSize up solved the issue. 

🙂

Quote

This raised an issue with an option that was required: 'SO_RCVBUF'. 

TWSocket.SetSocketRcvBufSize include that option. Has a look at the source code. The socket must be open to change that option since the socket handle is required.