Angus Robertson

Quote

http://welcometothe.中国

That works OK, but redirects to an Alibaba site using western domains. 

I did try and register an accented test domain earlier with 123-Reg.  My attempt for co.uk was declined but they registered a com OK except it is missing all the accented characters, won't accept xn-- names only Unicode.   I'll try an eu instead, they must support accents.

Angus

It seems those using non-English domains hedge their bets on their sites:

Handshake done, error #0 - SSL Connected OK with TLSv1.2, cipher ECDHE-RSA-AES128-GCM-SHA256, key auth RSA, key exchange ECDH, encryption AESGCM(128), message auth AEAD
! VerifyResult: ok, Peer domain: мособлеирц.рф
3 Certificate(s) in the verify chain.
#3 Issued to (CN): mosobleirc.ru
Alt Domains (SAN): mosobleirc.ru, www.mosobleirc.ru, www.мособлеирц.рф, www.новый.мособлеирц.рф, мособлеирц.рф, новый.мособлеирц.рф
Issued by (CN): Let's Encrypt Authority X3, (O): Let's Encrypt
Expires: 11/05/2020 18:43:06, Signature: sha256WithRSAEncryption
 

Does anyone have any working Far East web sites with IDNs, Chinese, Japanese, etc, those I've tried are all dead.

Angus

Testing IDN with Windows 2019 DNS Server is proving problematic.  I've been able to set-up A records for scrúdú and xn--scrd-srab,  I would have expected DNS Manager to convert an accented domain to puncode ASCII but it actually stores scr\303\272d\303\272 in the file.  It also stores 16-bit characters.  I had to convert the punycode version manually.

Old versions of ICS are actually able to lookup scrúdú.ftptest.co.uk without any punycode, so it seems windows usea the full 8-bits for DNS queries,  Internally, we convert Unicode to ANSI before the query, so this will only work for code pages that match the DNS server. 

The real question is whether this DNS behaviour is by design or ignorance, perhaps internal networks are allow full 8-bit local names?  I've Googled a lot, but can not find any design recommendations for IDN and Windows DNS Server, indeed no mentions atall. 

I was planning on changing ICS to automatically convert IDNs in Unicode to punycode ASCII, but this would break existing internal applications using 8-bit ANSI, so perhaps it needs to be optional, I'd prefer ASCII to become the default.

Also, think the HTTP client and server need changing, because the Host: header needs to be punycode ASCII, probably Location: as well, but sure about sub directories, arguments are always character converted but does that apply to directories as well? 

Angus

Sorry, really do not have time to explore 20 year old code,  ICS still supports Delphi 7 and later, but not earlier due to lack of language features. 

Angus

Quote

https://мособлеирц.рф/

Not a single ASCII character in that name, converts to xn--90aijkdmaud0d.xn--p1ai with no single hyphens, although conversion back to Unicode does not like it. 

IDN is now working with forward DNS lookups, using the DnsLookup function, now need to look at other places that don't use that function like ping.

Angus

It is a very strange format, no real reason for re-ordering the letters. 

Angus

ICS already has the inline MIME processing used for email headers since international names are far more common than international domains, bu this needs to be done at application level, the SMTP and POP3 components don't do it internally. 

Rather than use the Windows IDN APIs, I've trying some pascal code first, multi-platform. 

Angus

Thanks for the URLs everyone, I've also setup www.éxàmplê.ftptest.co.uk although it's not working yet, will make sure it works with SSL as a server as well.

Angus

Thanks everyone, I'll add IDN support to TWSocket.DnsLookup and TWSocket.ReverseDnsLookup which fortunately have string arguments.  I'll set-up a sub-domain with accents for testing.  Then think about email. 

If anyone has any real IDN URLs  they would be useful for testing. 

Angus

Do you ever get questions about IDN support in Indy, because I don't recall it ever being mentioned in the ICS mailing list.  No point adding IDN support if no-one is going to use, except the one Delphi developer that emailed me, not about ICS.

Angus

Has anyone have a requirement to support IDNs in ICS, or have any punycode conversion functions to share with ICS?

Someone just filled in a form on my web site with email domain with an accented character, first time I've seen that.  My ICS email client can handle the MIME encoded email header for display, but is unable to reply to the email due to no IDN handling.  I guess punycode conversion would fix that, but not sure which email header fields need which encoding.

Or indeed whether we should try and handle IDN at a lower level in ICS. 

Guess I should register an accented domain name for testing.

Angus

If you can do so easily, removing ReqVerHigh and ReqVerLow from the DFMs is safest, maybe an earlier release of Delphi stored default values or you set them to 1 after the default changed to 2, when is not noted in the unit  Can not see the winsock version changing again in my life time.  

The next low level change is likely to be QUIC which Google has been developing for several years, this uses UDP for TCP like streams, so should not involve winsock, it should be supported by OpenSSL within a year or so, once the spec is finalised. 

Angus

Not sure how that could happen, THttpCli does not reference ReqVerHigh or ReqVerLow and the underlying TWSocket is not accessible for writing in current versions.  ReqVerHigh or ReqVerLow would only be saved in the DFM if you changed them from the default of 2, again unless you are using an ancient version of ICS.

Angus

I think that property goes back to Windows XP or something when there were different versions of winsock.  But they currently default to 2 and need never be changed. 

You should only get that error if your application tries to change ReqVerLow after TWSocket has loaded, and why would you want to?

Angus

ICS includes a unit OverbyteIcsAvlTrees.pas written by Arno Garrels, from the unit:

Implements a fast cache-like data storage based on two linked AVL-Trees for primary and secondary indexing.  Primary key of type string has to be unique, secondary key of type TDateTime may have duplicates.   AVL or balanced binary trees are extremely efficient data structures for searching data. Finding an element in  65536 requires at most 16 compares.  Uses an AVL-Tree as it is described in the book "Algorithms & Data Structures", Prof. Niklaus Wirth.

No real dependencies on other ICS units.

Angus

You can apply file permissions to Windows folders against Windows logins, but not password protect a folder. 

Yes, ICS is from Overbyte, there is an ICS forum.  You could run your FTP server on strange ports, but that is usually a nightmare with FTP and firewalls/routers.  Fortunately my hosted server has a /27 subnet, don't even use all of them.

Angus

IIS FTP uses Windows accounts, so you need to set-up those from Delphi as well, can probably be scripted. 

The usual solution to your problem is to use a long random file name that disappears once the download is completed, but you need to watch the log to see when that happens.

The better solution is to you use a Delphi FTP server, like the ICS one I support, then you can control logins and directories yourself easily, using the ICS FTP Server sample you should have a working solution in a few hours, days faster than using WMI.  But it needs it's own IP address and port, will not help if you have to use IIS.  

There is no type library for the IIS management stuff, it's all WBEM based code.

Angus

You can start reading at https://www.iis.net/overview/control/powerfuladmintools

I'm just looking at WMI scripting for controlling IIS.  Setting up a virtual directory through IIS admin is easy.

Angus

Think that got broken during the integration into ICS, the original TMagFileCopy version had a messagepump that used fMultiThreaded, but I changed the class from TComponent to TIcsWndControl and did not change use of MultiThreaded.  Will fix it shortly.

Angus

Yes, I've been using SvCom for 15 years, but it is commercial software.  DDService actually patches your local copy of TService, while SvCom borrowed it. 

Angus

Don't think monitor power off will really help you, you need to check PowerEvents like PBT_APMQUERYSUSPEND and PBT_APMRESUMESUSPEND.  I'll email my unit that does all this.

Angus

HandlexEx is supported by the DDService replacement for the Delphi version, written by Arno Garrels. 

You can also intercept WMPowerBroadcast messages which is sent at various stages before and after suspend and works fine in a service.  I have a THardwareEvents component I've never got around to releasing due to lack of documentation.

Angus

I have seen DNS amplification attacks using my DNS in the past, usually from the size of the firewall logs, but then block it using the external firewall.  Generally I'm not too worried about exploits, no-one has ever successfully attacked my servers. 

I have however moved the DNS for a test domain to Cloudfare, so will add updating it's DNS records to the ICS sample application alongside Windows DNS,  Not sure if I also moved the web site to Cloudfare, the dashboard is very confusing and I don't really care at the moment, need to write code instead.

Angus

My public DNS servers have recursion and caching disabled, they are primary/secondary DNS servers, not used for local DNS.  I believe that avoids the worst abuse, but certainly not an expert.

My experience of the three Let's Encrypt challenge methods is they are all similar speed, The ICS component already does DNS, it tells you what TXT records to set-up manually, I'm just making it easier. 

Now if someone has a Delphi component that handles the various Cloud DNS provides APIs I'd love to use it.  I'm sure it's not complicated, just time consuming to set-up accounts with various providers to test it.

Angus

I get the impression from reading the windns.h DNS API documentation that it's mainly for querying and modifying caching DNS servers, rather than updating primary DNS servers, no functions for server setup,  zones, etc, all of which are in the DNS WMI API.  I also need this to work over a LAN, which WMI handles, albeit sluggishly.

Angus