merijnb

We have many projects, all with a huge list of search paths defined. Many of these search paths are identical. Once a path needs to be added to this, we need to update the search path on many projects, which is a pita.

I'm now experimenting with putting all these paths in an environment variable, this way, I only need to update the environment variable, use that environment variable in the search path of the project and it's done.

This all works well and looks like this:
set MY_DELPHI_PATHS = 'lots and lots of paths'

in Delphi set this in the search path of the project configuration:
$(MY_DELPHI_PATHS)

but unfortunately both the Open File expert and the Uses Clause Manager seem unable to cope with this. The uses clause manager shows the files in these search paths in the 'project' tab, but not in the 'search path' tab, the Open Files Expert doesn't show the files in these search paths anywhere.

Can these editors be made environment variable aware?

Thanks!

I've just noticed the feature request list on sourceforge, so I've added this there as well

57 minutes ago, Angus Robertson said:

The safest way to read bundle files with unicode characters will be to strip out all the comments I add before passing them to OpenSSL. 

 

For maintenance, adding certificate fields makes life easy, I hate long blocks of base64, and previously OpenSSL has seemed good at ignoring added text.  But if the comments are causing trouble, they should go.

 

Angus

 

Or just convert it to ascii of unicode gives the issues

4 minutes ago, Kas Ob. said:

I saw something very similar to this but it wasn't OpenSSL but different libraries, the reason for this wrong parsing is coming from Language for Non-Unicode program settings in Windows Language settings, you can confirm if this is the case by finding what settings on those few machines, away from that removing those characters might help but the better solution is to convert this bundle file to Unicode or UTF8 ,...(whatsoever) as long you don't leave it to Windows and OpenSSL.

I can confirm that this checkbox was checked, after unchecking and rebooting the problem is gone.

note: We have seen these issues with multiple versions of the OpenSSL binaries, including the ones bundled with ICS 8.64.

Please consider the following code snippet:

 var s := TSslContext.Create(nil);
 try
  s.SslCALines.Text := sslRootCACertsBundle();
  try
   s.InitContext();
  except
   on e: exception do
    ShowMessage(e.Message);
  end;
 finally
  s.Free();
 end;

This snippet works on most machines, but we have a few machines at client sites, where this code raises an exception: 'Error reading info file "Lines"'. (We haven't figured out yet why these specific machines have this issue).

This exception is raised on OverbyteIcsWSocket.pas:14667 (ICS 8.64).

    InfoStack := PStack(f_PEM_X509_INFO_read_bio(InBIO, nil, nil, nil));
    if not Assigned(InfoStack) then
        raise ESslContextException.CreateFmt('Error reading info file "%s"', [FileName]);

Apparently the call to f_PEM_X509_INFO_read_bio() failed, but no information is give why, so we've added some calls to f_ERR_get_error() to see what is going wrong, and then it turns out the call to f_PEM_X509_INFO_read_bio() sets the error code 151580774 (get_header_and_data).

To find out which of the hard coded certificates returned by sslRootCACertsBundle() were making this error happening, we've passed each of the certificates (sslRootCACerts001, sslRootCACerts002, etc) one by one to an TSSLContext to see which ones failed. This showed 2 interesting things:

There is a 'typo' in sslRootCACerts009 and sslRootCACerts011:
 

        'AGAT/3B+XxFNSRuzFVJ7yVTav52Vr2ua2J7p8eRDjeIRRDq/r72DQnNSi6q7pynP' + #13#10 +
        '9WQcCk3RvKqsnyrQ/39/2n3qse0wJcGE2jTSW3iDVuycNsMm4hH2Z0kdkquM++v/' + #13#10 +
        'eu6FSqdQgPCnXEqULl8FmTxSQeDNtGPPAUO6nIPcj2A781q0tHuu2guQOHXvgR1m' + #13#10 +
        '0vdXcDazv/wor3ElhVsT/h5/WrQ8' + #13#10;
    sslRootCACerts010 =
        '-----END CERTIFICATE-----' + #13#10 +
        '# X509 SSL Certificate' + #13#10 +

and

        'S0Zj/gA0QHDBw7mh3aZw4gSzQbzpgJHqZJx64SIDqZxubw5lT2yHh17zbqD5daWb' + #13#10 +
        'QOhTsiedSrnAdyGN/4fy3ryM7xfft0kL0fJuMAsaDk527RH89elWsn2/x20Kk4yl' + #13#10 +
        '0MC2Hb46TpSi125sC8KKfPog88Tk5c0NqMuRkrF8hey1FGlmDoLnzc7ILaZRfyHB' + #13#10 +
        'NVOFBkpdn627G190' + #13#10;
    sslRootCACerts012 =
        '-----END CERTIFICATE-----' + #13#10 +
        '# X509 SSL Certificate' + #13#10 +

(the END CERTIFICATE line is part of the wrong constant), which isn't a problem when you call sslRootCACertsBundle() since all constants are concatenated, but which shows up if you try to use these certificates separately.

More interestingly, one these specific machines there seems to be an issue with sslRootCACerts033 and sslRootCACerts034.

This is sslRootCACerts033:

# X509 SSL Certificate
# Subject Common Name: GlobalSign
# Subject Organisation: GlobalSign
# Subject Organisation Unit: GlobalSign Root CA - R2
# GlobalSign Root R2 SHA1 • RSA • 2048
# Issuer: Self Signed
# Expires: 15/12/2021
-----BEGIN CERTIFICATE-----
MIIDujCCAqKgAwIBAgILBAAAAAABD4Ym5g0wDQYJKoZIhvcNAQEFBQAwTDEgMB4G
A1UECxMXR2xvYmFsU2lnbiBSb290IENBIC0gUjIxEzARBgNVBAoTCkdsb2JhbFNp
Z24xEzARBgNVBAMTCkdsb2JhbFNpZ24wHhcNMDYxMjE1MDgwMDAwWhcNMjExMjE1
MDgwMDAwWjBMMSAwHgYDVQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMjETMBEG
A1UEChMKR2xvYmFsU2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjCCASIwDQYJKoZI
hvcNAQEBBQADggEPADCCAQoCggEBAKbPJA6+Lm8omUVCxKs+IVSbC9N/hHD6ErPL
v4dfxn+G07IwXNb9rfF73OX4YJYJkhD10FPe+3t+c4isUoh7SqbKSaZeqKeMWhG8
eoLrvozps6yWJQeXSpkqBy+0Hne/ig+1AnwblrjFuTosvNYSuetZfeLQBoZfXklq
tTleiDTsvHgMCJiEbKjNS7SgfQx5TfC4LcshytVsW33hoCmEofnTlEnLJGKRILzd
C9XZzPnqJworc5HGnRusyMvo4KD0L5CLTfuwNhv2GXqF4G3yYROIXJ/gkwpRl4pa
zq+r1feqCapgvdzZX99yqWATXgAByUr6P6TqBwMhAo6CygPCm48CAwEAAaOBnDCB
mTAOBgNVHQ8BAf8EBAMCAQYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUm+IH
V2ccHsBqBt5ZtJot39wZhi4wNgYDVR0fBC8wLTAroCmgJ4YlaHR0cDovL2NybC5n
bG9iYWxzaWduLm5ldC9yb290LXIyLmNybDAfBgNVHSMEGDAWgBSb4gdXZxwewGoG
3lm0mi3f3BmGLjANBgkqhkiG9w0BAQUFAAOCAQEAmYFThxxol4aR7OBKuEQLq4Gs
J0/WwbgcQ3izDJr86iw8bmEbTUsp9Z8FHSbBuOmDAGJFtqkIk7mpM0sYmsL4h4hO
291xNBrBVNpGP+DTKqttVCL1OmLNIG+6KYnX3ZHu01yiPqFbQfXf5WRDLenVOavS
ot+3i9DAgBkcRcAtjOj4LaR0VknFBbVPFd5uRHg5h6h+u/N5GJG79G+dwfCMNYxd
AfvDbbnvRG15RjF+Cv6pgsH/76tuIMRQyV+dTZsXjAzlAcmgQWpzU/qlULRuJQ/7
TBj0/VLZjmmx6BEP3ojY+x1J96relc8geMJgEtslQIxq/H5COEBkEveegeGTLg==
-----END CERTIFICATE-----

When processing this certificate on these specific machines OpenSSL gives this error, unless:

- I remove one of the • in the 5th line from the top. (As long as there is only 1 • in the file it's ok, as soon as I add a 2nd • anywhere in the header the error returns)

- I add a CRLF at the bottom of the file

The same thing applies to sslRootCACerts034. I realize that this is probably a bug in OpenSSL itself, but it might be wise to remove the •'s from the hard coded certificates to work around this issue.

Curious if anyone can shed some light on the cause of this 🙂

Merijn

10 minutes ago, Angus Robertson said:

If multicast needs to listen on multiple UDP ports and/or addresses, just create an array of TWSocket components with different IP addresses/ports for each, sharing the same event handlers.  For TCP we already have TMultiListenWSocketServer which is the same as TWSocketServer so web servers listen on lots of ports and IPv4 and IPv6 at the same time.

 

Angus

The issue currently is not listening on multiple ports and / or addresses, receiving multicast data on a single ip and port doesnt' work either. It seems to be with the fact that you need to register with the kernel what unicast groups you want to listen to, otherwise the datagrams aren't even passed to your process.

Hey Dave,

I'm not going to switch from ICS to Indy, but with your experience with getting receiving multicast, can you see if there is anything wrong with the snippet from ICS above, or can you see if there is anything missing? Is calling setsocktopt with IP_ADD_MEMBERSHIP what needs to be done?

38 minutes ago, Angus Robertson said:

Sorry can not really help any further, I think my last UDP project was just broadcasting not multicasting. 

 

I would test keeping things simple, ignore multicast completely, just a simple UDP listener and see what happens.  Your can use the OverbyteIcsIpStmLogTst sample to send normal UDP packets to make sure your server is receiving something, then look into why multicast is not working.

 

Angus

UDP singlecast and broadcast is working fine (I've done that many times), this is the first time I need to use multicast though and it behaves a little bit differently internally (the whole IP_ADD_MEMBERSHIP thing).

I respect you (Angus) can't help me though, any other takers? Or do we know who wrote this part in ICS originally?

9 hours ago, Remy Lebeau said:

In addition to what has already been said, 255.255.255.255 is not a valid Multicast group IP, it is a UDP subnet broadcast IP.  Subnet broadcasts are very different than Multicast broadcasts.  Valid multicast group IPs (for IPv4) are in the range of 224.0.0.0 to 239.255.255.255.

Hi Remy,

Thanks, but I realize that, but the multicast group ip being used is 225.255.255.255 (easy to overlook that 🙂 )

3 hours ago, Angus Robertson said:

TWSocketServer is only for TCP, not UDP which does not have a concept of clients or connections.  Change it to TWSocket. 

 

Not used multi casting for many years, but suspect that MultiCastAddrStr is for sending stuff, not receiving it. 

 

V8.60 added a new component TIcsIpStrmLog with a sample OverbyteIcsIpStmLogTst.dpr which has a logprotUdpServer mode that allows you to create a server with little code handling receiving data for you.

 

Angus

Tx for the wonderful quick reply. I've looked at TIcsIpStrmLog but there is no use of multicast in there.

With regard to MultiCastAddrStr being for sending only I'm quite sure it's not.

I've found several places where it's mentioned that to use multicast listening it's required to call setsockopt() with IP_ADD_MEMBERSHIP to 'register' the multicast group you want to listen to with the kernel, the working piece of Python code also does something similar. Other places this is mentioned are for example:

https://stackoverflow.com/questions/43322462/multicast-packet-seen-in-wireshark-but-not-received-by-user-program-windows-an

and

http://www.tldp.org/HOWTO/Multicast-HOWTO-6.html

There is something in the ICS wiki about receiving multicast data:

 WS.Proto:='udp';
 WS.Addr:='192.168.0.1'; // IP address of the physical adapter I want to bind to
 WS.Port:='5000';  
 WS.MultiCast:=true;
 WS.ReuseAddr:=true;
 WS.MultiCastAddrStr:='239.192.1.1'; // Mutlicast IP  
 WS.Listen;

(from http://wiki.overbyte.eu/wiki/index.php/TWSocket.MultiCast)

I also see something like that in OverbyteIcsWsocket:

            if FMultiCast then begin
                if FAddrFormat = AF_INET then begin
                    { Use setsockopt() to join a multicast group }
                    { mreq.imr_multiaddr.s_addr := WSocket_inet_addr('225.0.0.37');}
                    { mreq.imr_multiaddr.s_addr := sin.sin_addr.s_addr;}
                    { mreq.imr_multiaddr.s_addr := WSocket_inet_addr(FAddrStr);}
                    mreq.imr_multiaddr.s_addr := WSocket_Synchronized_inet_addr(PAnsiChar(AnsiString(FMultiCastAddrStr)));
                    { mreq.imr_interface.s_addr := htonl(INADDR_ANY);} { RK}
                    mreq.imr_interface.s_addr := WSocket_Synchronized_ResolveHost(AnsiString(FAddrStr)).s_addr;
                    iStatus := WSocket_Synchronized_SetSockOpt(FHSocket, IPPROTO_IP,
                                                               IP_ADD_MEMBERSHIP,
                                                               @mreq, SizeOf(mreq));

                    if iStatus <> 0 then begin
                        SocketError('setsockopt(IP_ADD_MEMBERSHIP)');
                        Exit;
                    end;

So I'm quite sure that it's required for receiving multicast data, but for some reason I'm not receiving anything, as if the implementation in the last snippet has something wrong.

Do we know who is the author of this snippet?

Thanks again!

I'm trying to receive UDP packets which are sent to a multicast group. I can see the packets being received in WireShark (dest ip = 225.255.255.255 dest port = 1120), and can receive the data using an example python script I found on the net, meaning that UDP multicast group and port are correct.

I'm creating an UDP server like this:

var UDPServer: TWSocketServer;
begin
 UDPServer := TWSocketServer.Create(Self);
 UDPServer.Proto := 'udp';
 UDPServer.Addr := '0.0.0.0';
 UDPServer.Port := '1120';

 UDPServer.MultiCast := true;
 UDPServer.ReuseAddr := true;
 UDPServer.MultiCastAddrStr := '225.255.255.255';

 UDPServer.OnDataAvailable := OnUDPDataAvailable;

 UDPServer.Listen();
end;

The OnUDPDataAvailable is never called, am I missing something here in setting the server up properly to listen for this group?