Angus Robertson

I added your change to my copy, not in SVN yet. Angus

Thanks, now fixed. Angus

Has anyone have a requirement to support IDNs in ICS, or have any punycode conversion functions to share with ICS? Someone just filled in a form on my web site with email domain with an accented character, first time I've seen that. My ICS email client can handle the MIME encoded email header for display, but is unable to reply to the email due to no IDN handling. I guess punycode conversion would fix that, but not sure which email header fields need which encoding. Or indeed whether we should try and handle IDN at a lower level in ICS. Guess I should register an accented domain name for testing. Angus

SVN and the overnight zip have been updated with a lot of changes so ICS supports International Domain Names for Applications (IDNA), i.e. using accents and Unicode characters in domain names. Domain names can only contain lowercase ASCII letters and numbers and a couple of symbols, so Unicode U-Labels (nodes in a domain) must be converted to A-Labels (Punycode ASCII) with an ACE (ASCII Compatible Encoding) prefix. So www.mâgsÿstést.eu becomes www.xn--mgsstst-pwa1e4l.eu and мособлеирц.рф becomes xn--90aijkdmaud0d.xn--p1ai. ICS mostly does the Unicode to A-Label conversion just before looking up an IP address for a domain name (in DnsLookup) and converts back from A-Label to Unicode when doing a reverse lookup (in ReverseDnsLookup). HTTP headers also contain A-Labels for the Host: header and the host part of URLs for proxy or relocation, but Unicode paths should be UrlEncoded by the application as now. Not looked at SMTP yet. The HTTP client and server, Ping, ICMP and DNS Query components all now support Unicode domain names, generally without application changes unless you want to display the A-Label name looked-up (PunycodeHost property). DNS Query does require application changes due to all methods and properties previously being AnsiString, now String. SSL/TLS now fully supports Unicode domain names, including displaying the Unicode version of the domain name (except for Subject and Issuer lines), and X509 automatic certificate ordering from Let's Encrypt fully supports Unicode domain names. Certificate files are saved with Unicode names, not A-Labels. For server testing I registered an eu domain which is live on one of my web sites at https://www.mâgsÿstést.eu/ and https://scrúdú.mâgsÿstést.eu/ which have ICS ordered SSL certificates. I do have DNS for Cyrillic and Far East domains, but this web server is built with Delphi 2007 so no full Unicode. Angus

That works OK, but redirects to an Alibaba site using western domains. I did try and register an accented test domain earlier with 123-Reg. My attempt for co.uk was declined but they registered a com OK except it is missing all the accented characters, won't accept xn-- names only Unicode. I'll try an eu instead, they must support accents. Angus

It seems those using non-English domains hedge their bets on their sites: Handshake done, error #0 - SSL Connected OK with TLSv1.2, cipher ECDHE-RSA-AES128-GCM-SHA256, key auth RSA, key exchange ECDH, encryption AESGCM(128), message auth AEAD ! VerifyResult: ok, Peer domain: мособлеирц.рф 3 Certificate(s) in the verify chain. #3 Issued to (CN): mosobleirc.ru Alt Domains (SAN): mosobleirc.ru, www.mosobleirc.ru, www.мособлеирц.рф, www.новый.мособлеирц.рф, мособлеирц.рф, новый.мособлеирц.рф Issued by (CN): Let's Encrypt Authority X3, (O): Let's Encrypt Expires: 11/05/2020 18:43:06, Signature: sha256WithRSAEncryption Does anyone have any working Far East web sites with IDNs, Chinese, Japanese, etc, those I've tried are all dead. Angus

Testing IDN with Windows 2019 DNS Server is proving problematic. I've been able to set-up A records for scrúdú and xn--scrd-srab, I would have expected DNS Manager to convert an accented domain to puncode ASCII but it actually stores scr\303\272d\303\272 in the file. It also stores 16-bit characters. I had to convert the punycode version manually. Old versions of ICS are actually able to lookup scrúdú.ftptest.co.uk without any punycode, so it seems windows usea the full 8-bits for DNS queries, Internally, we convert Unicode to ANSI before the query, so this will only work for code pages that match the DNS server. The real question is whether this DNS behaviour is by design or ignorance, perhaps internal networks are allow full 8-bit local names? I've Googled a lot, but can not find any design recommendations for IDN and Windows DNS Server, indeed no mentions atall. I was planning on changing ICS to automatically convert IDNs in Unicode to punycode ASCII, but this would break existing internal applications using 8-bit ANSI, so perhaps it needs to be optional, I'd prefer ASCII to become the default. Also, think the HTTP client and server need changing, because the Host: header needs to be punycode ASCII, probably Location: as well, but sure about sub directories, arguments are always character converted but does that apply to directories as well? Angus

Sorry, really do not have time to explore 20 year old code, ICS still supports Delphi 7 and later, but not earlier due to lack of language features. Angus

Not a single ASCII character in that name, converts to xn--90aijkdmaud0d.xn--p1ai with no single hyphens, although conversion back to Unicode does not like it. IDN is now working with forward DNS lookups, using the DnsLookup function, now need to look at other places that don't use that function like ping. Angus

It is a very strange format, no real reason for re-ordering the letters. Angus

ICS already has the inline MIME processing used for email headers since international names are far more common than international domains, bu this needs to be done at application level, the SMTP and POP3 components don't do it internally. Rather than use the Windows IDN APIs, I've trying some pascal code first, multi-platform. Angus

Thanks for the URLs everyone, I've also setup www.éxàmplê.ftptest.co.uk although it's not working yet, will make sure it works with SSL as a server as well. Angus

Thanks everyone, I'll add IDN support to TWSocket.DnsLookup and TWSocket.ReverseDnsLookup which fortunately have string arguments. I'll set-up a sub-domain with accents for testing. Then think about email. If anyone has any real IDN URLs they would be useful for testing. Angus

Do you ever get questions about IDN support in Indy, because I don't recall it ever being mentioned in the ICS mailing list. No point adding IDN support if no-one is going to use, except the one Delphi developer that emailed me, not about ICS. Angus

If you can do so easily, removing ReqVerHigh and ReqVerLow from the DFMs is safest, maybe an earlier release of Delphi stored default values or you set them to 1 after the default changed to 2, when is not noted in the unit Can not see the winsock version changing again in my life time. The next low level change is likely to be QUIC which Google has been developing for several years, this uses UDP for TCP like streams, so should not involve winsock, it should be supported by OpenSSL within a year or so, once the spec is finalised. Angus

Not sure how that could happen, THttpCli does not reference ReqVerHigh or ReqVerLow and the underlying TWSocket is not accessible for writing in current versions. ReqVerHigh or ReqVerLow would only be saved in the DFM if you changed them from the default of 2, again unless you are using an ancient version of ICS. Angus

I think that property goes back to Windows XP or something when there were different versions of winsock. But they currently default to 2 and need never be changed. You should only get that error if your application tries to change ReqVerLow after TWSocket has loaded, and why would you want to? Angus

ICS includes a unit OverbyteIcsAvlTrees.pas written by Arno Garrels, from the unit: Implements a fast cache-like data storage based on two linked AVL-Trees for primary and secondary indexing. Primary key of type string has to be unique, secondary key of type TDateTime may have duplicates. AVL or balanced binary trees are extremely efficient data structures for searching data. Finding an element in 65536 requires at most 16 compares. Uses an AVL-Tree as it is described in the book "Algorithms & Data Structures", Prof. Niklaus Wirth. No real dependencies on other ICS units. Angus

You can apply file permissions to Windows folders against Windows logins, but not password protect a folder. Yes, ICS is from Overbyte, there is an ICS forum. You could run your FTP server on strange ports, but that is usually a nightmare with FTP and firewalls/routers. Fortunately my hosted server has a /27 subnet, don't even use all of them. Angus

IIS FTP uses Windows accounts, so you need to set-up those from Delphi as well, can probably be scripted. The usual solution to your problem is to use a long random file name that disappears once the download is completed, but you need to watch the log to see when that happens. The better solution is to you use a Delphi FTP server, like the ICS one I support, then you can control logins and directories yourself easily, using the ICS FTP Server sample you should have a working solution in a few hours, days faster than using WMI. But it needs it's own IP address and port, will not help if you have to use IIS. There is no type library for the IIS management stuff, it's all WBEM based code. Angus

You can start reading at https://www.iis.net/overview/control/powerfuladmintools I'm just looking at WMI scripting for controlling IIS. Setting up a virtual directory through IIS admin is easy. Angus

Think that got broken during the integration into ICS, the original TMagFileCopy version had a messagepump that used fMultiThreaded, but I changed the class from TComponent to TIcsWndControl and did not change use of MultiThreaded. Will fix it shortly. Angus

Yes, I've been using SvCom for 15 years, but it is commercial software. DDService actually patches your local copy of TService, while SvCom borrowed it. Angus

Don't think monitor power off will really help you, you need to check PowerEvents like PBT_APMQUERYSUSPEND and PBT_APMRESUMESUSPEND. I'll email my unit that does all this. Angus

HandlexEx is supported by the DDService replacement for the Delphi version, written by Arno Garrels. You can also intercept WMPowerBroadcast messages which is sent at various stages before and after suspend and works fine in a service. I have a THardwareEvents component I've never got around to releasing due to lack of documentation. Angus