Angus Robertson

OverbyteIcsWebAppServerUploads,pas is a handler that accepts POSTed files. Angus

No TSimpleWebSrv does not currently support POST or PUT, if you look at the diagnostic log output you'll see 'Server Request Ignored'. It was designed as a light weight server for OAUTH2 and responding to well-known requests by certificate authorities. Adding complexity increases the size of any ICS REST applications that use those features. Angus

This list is possible additions to ICS, new protocols and functionality, none of which is guaranteed... Open to suggestions for other possible protocol additions or improvements. Personally, I'm unlikely to look at any of this stuff for several months, unless my company has an urgent need for something new. But if several other users are all looking for the same thing, I can help co-ordinating improvements. Protocol: STUN client and server Why: Session Traversal Utilities for NAT allows finding a public IP address while behind a NAT router, by contacting a STUN server. Used by public servers and clients that need to tell other applications how to contact them. Also some client protocols like a host name, like SMTP. Difficulty: low, simple protocol, easy to implement. Benefits: medium, saves configuring the IP manually. Protocol: RDAP Client Why: Registration Data Access Protocol is the replacement for the Whois protocol, using HTTPS REST and Json protocols. Both domains and IP addresses. Difficulty: low, simple protocol, easy to implement. Benefits: low, Whois is heavily censored now. Protocol: Roughtime client and server Why: replacement for NTP and SNTP network time protocols (from Google), with security. Difficulty: low, simple protocol. Benefits: low, usually get time from Windows. Protocol: HTTP/2 for HTTP client and server Why: More efficient version of HTTP/1.1, particularly for web pages with dozens of elements, compresses headers. Difficult: moderate, extra DLL, messy, lots to change. Benefits: low, ICS is rarely used to download complex web pages, perhaps more important for the HTTP server. Protocol: SASL for SMPT and POP3 clients Why: Simple Authorisation and Security Layer adds OAuth2 for SMTP and POP3, safer than clear authentication. Difficulty: low, OAuth2 already done. Benefits: high, where the email provider requires it. Protocol: OAuth1 for Twitter Why: Twitter uses OAuth1 rather than the easier and more recent OAuth2 almost everyone else uses. Difficulty: low, uses HMAC which is done already. Benefits: high, if you want to send tweets. Protocol: DNS over HTTPS (DOH) Why: secure DNS can not be intercepted and modified. Difficulty: ICS already has a TDnsQueryHttps component and sample, but causing it to be used by TWSocket and other components at low level could get messy and link in all the REST and Json units. Benefits: low, Microsoft is threatening to support DOH, probably only Windows 10/2019. Protocol: MQTT Why: MQ Telemetry Transport is used to send messages between devices, including IoT. Difficulty: ICS MQTT project n GutHub, needs integration. Benefits: high, if you need the protocol. Protocol: Websockets server Why: A full duplex version of HTTP often used for server push to dynamically update web pages. Difficulty: ICS websockets project in GutHub, needs update and integration. Benefits: high, if you need the protocol. Protocol: SChannel SSL/TLS Support Why: avoid distributing OpenSSL DLLs by using SSL/TLS protocol APIs built into Windows. Downside is Microsoft often takes years to support new protocols and often only in the latest operating systems Difficulty: high, needs to be done at the lowest levels, risks adding bugs for OpenSSL if both supported, need to replace a lot of OpenSSL encryption APIs with Windows APIs, and certificate APIs. Benefits: low, unless you really hate DLLs. Protocol: POP3 Server, IMAP Client and Server Why: because these are missing and we all use email. Difficulty: moderate, lot of new new code. Benefits: high, if you need them. Platform: Better C++ and MacOS Support Why: we don't do much testing on C++ and MacOS due to lack of volunteers to do this regularly. We lack samples for C++ and MacOS. Difficulty: high, users want someone else to do the work. Benefits: high, for C++ and MacOS users. Platform: Support for mobile apps and Linux Why: more platforms. Difficulty: very high, probably at least one man year effort, maybe more. Benefits: high, for mobile apps and Linux. Angus

Better use of multiple CPUs needs threads, which we generally try to avoid in ICS, but bring the ICS threaded socket server up to date is on my list, and perhaps allowing the web server to use it. I'm aware there are Windows APIs that offer performance improvements over our existing use of sockets, by sending and receiving entire files, but you then lose control over what is happening and SSL becomes problematic. But it's something we can look at. Angus

I've explained the limitations of the TCP implementation on Windows and how to get around it. If you don't want to listen to my experience, feel free to Google for better solutions then let us know when you have a proven working solution. Many of us have done all this before, often several times. Angus

I have already given my suggestions on how to solve very common issue. If you need new features you have the ICS source and you can update it yourself. Or listen to those of us that have been using it for 20 years. Angus

If you want to write your own internet library for your own requiremets, please feel free. Angus

I would suggest reading ICS wiki FAQ_Using_TSslCertTools which explains all about certificates, and FAQ_Order_SSL_Certificates. There are other FAQ on how we handle trust stores. Angus

TCustomTimeoutWSocket timeouts relate to a socket that is already opened and connected, primarily a server socket, not to TCP DNS and connection attempts. As Francois says, you can use a timer to abort the connection, but this does not stop the windows connection attempt so the socket can not be re-used immediately for another attempt. Angus

Those are really old files and should have been removed a long time ago. The primary demo and documentation for X509 certificates is OverbyteIcsPemTool.dpr which will display, create, and convert numerous certificate formats, it is complex but will do almost everything the OpenSSL command line tool will do, in code. For a simple self signed certificate, there is an ICS function CreateSelfSignedCert in OverbyteIcsSslX509Utils. Angus

I assume you are now talking about clients and not servers? Simple answer is no, TCP is a lazy protocol based on waiting for responses from slow networks (decades ago) so nothing happens fast. Long answer is Windows allows you to change the TCP timeouts, but such changes effect all applications on the PC and generally it's a very bad idea. Real solution is to try and ping the server before connecting using TCP, because you can timeout after a few seconds and give up. ICS has a threaded ping component for this purpose. and a high level client/server component TIcsIpStrmLog that uses ping for exactly this purpose. TIcsIpStrmLog may be configured as a client or server and allows simple text lines to sent, using few lines of code in the application and only a couple of events. There is a sample OverbyteIcsIpStmLogTst.dpr that send packets to itself. The component supports SSL and will order it's own certificates. Only problem with ping is sometimes it's blocked by firewalls so no response. You can use a UDP server in parallel with the TCP server for a quick hello, no response is similar to ping. Angus

The ICS FTP server and client already support One Time Passwords automatically, added that long before SSL/TLS became common, In the server, set the OtpMethod to those you wish to iupport, and the server hello prompt changes to advertise OTP, which the FTP client handles automatically. Angus

SVN and the overnight zip are updated with the 64-bit fix. Still having fun and games with 4-bit samples, think I'll try an older version of Delphi. I checked other functions with TBytes, most already had const or var, a couple were missing both so added, although they were only used in ICS with TBytes buffers so should not have caused any issues. Angus

Might come back this in the summer when OpenSSL 3.0 adds support for Certificate Management Protocol, which I'll try to support in ICS as well as Let's Encrypt. Angus

I've only been talking about server certificates here. Client certificates are totally different and make things very confusing. ICS clients and servers do support client certificates but not automatically and there are no samples using them, really only needed for exceptionally high security stuff like Swift bank transfers. The ICS component has properties to add digitalSignature and codeSigning extensions to certificates, not sure which OID they add. Angus

If you are talking about a simple analogue telephone line, this is all down to the hardware monitoring that line and how it delivers the caller id to your computer., 20 years ago some modems and special hardware had this feature, and you use an RS232 port to read it on a PC. But with analogue telephony being rapidly replaced by VoIP and the old PSTN copper cable telephone networks and exchanges turned off and obsoleted (by 2025 in the UK), this is really historical curiosity. Angus

The latest version of the ICS FTP server includes full SSL/TLS support and will even automatically order and install free Let's Encrypt certificates, or commercial certificates if you really want to pay. The most important point about certificates is they are mostly for public domain names that can be accessed on the public internet, and the certificates are published in public logs for any to find and check. Indeed hackers monitor those logs, and within a few seconds of a new certificate being issued expect hacking attempts to port 80. BTW, port 80 needs to be open to order a domain validated certificate. and the ICS FTP server also has an HTTP server to satisfy the request, only runs for a few seconds until the order process is finished. If you don't care about domain validation and only encryption, you can use self signed certificates which ICS will also generate. The X509Certs component will even issue certificates signed by a private CA, I use that on my LAN for testing lots of internal servers. Angus

So reference counting was the culprit, sorry missed that in the blizzard of overnight messages. I don't use TBytes very often but will carefully check ICS for all such use. As I said yesterday, the web server sample was erratic under Win64, but my development PC had not been rebooted since the last patch Tuesday and does become more unstable over time. Should be able to update SVN later today. Angus

Making it const Buffer: TBytes; is a safe change, var would be dangerous since it's a cast pointer. Still confused why it would be necessary, I always thought pointers did not need to be declared const, and the different behaviour when compiled with Win32 and Win64 suggests a compiler bug to me? I'll need to check a few other functions using TBytes. Angus

Since I can not reproduce the problem with Win64, I'll do nothing more for now. I still await the IcsLogger log lines I asked for. Angus

His solution may have fixed the exception, but will not return any sensible ALPN data, and will probably introduce other problems instead. I did wonder whether I should use PAnsiChar instead of TBytes, which would be safer in case someone in the future tried to resize the parameter. But TBytes seems to be the preferred method for working with binary and is mobile compatible, where Ansi is not. Angus

I quite understand that pointers differ in Win64, but they that code is not copying pointers, it is copying 8-bit bytes. Will you please build the OverbyteIcsSslWebServ sample with ICS 8.64 Win64 and your changes, make an request to https://localhost/demo.html with Display SSL Info and Logger Dest Event ticked, the log should contain lines with AlpnCB similar to: [16:22:53 127.0.0.1] SNI "localhost" received 16:22:53:024 AlpnCB> inlen: 12 - 02683208687474702F312E31 16:22:53:031 AlpnCB> Protocols: h2,http/1.1 [16:22:53] SSL Application Layer Protocols allowed from client: h2,http/1.1 16:22:53:044 0000000002710020 ICB> SSL_accept: SSLv3/TLS read client hello where those two lines are the input and output from IcsWireFmtToStrList. I'm not going to look at this further with evidence of a real problem in ICS. Or you could add IcsLogger to your own server to get the same logging information. Angus

Sorry, but you misunderstand the declaration for TBytes, it is a pointer to a dynamic array of bytes, not a pointer to a array of pointers. A byte remains the same size however compiled, so your proposed suggestion of copying the content based on the pointer size will cause many problems. I can only assume your correction was theoretical and you did not test it with an ICS SSL server application that uses the code. I've just built the OverbyteIcsSslWebServ sample with Win64, and ALPN is working as expected, as I showed earlier in this thread, I am getting some other strange exceptions running Win64 samples today, so there may be other Win64 issues elsewhere, or with OpenSSL Win64, but not with ALPN. Don't have time to investigate further at the moment. Angus

The ALPN changes are in V8.64 which is not released yet, but the changes can be downloaded from the SVN overnight zip on the page you mention. However I did not fix any bugs in the server version, just improved the code as previously mentioned. Angus

If you are using TSslWSocketThrdServer then you should be using TSslWSocketThrdClient. However TSslWSocketThrdServer is itself a descendent of TSslWSocketServer which does not use threads, so it all depends on how you are using the component. OnSessionClosed is only gone to triggered once since the client is destroyed immediately afterwards. Presumably you started with the OverbyteIcsThrdSrvV3.dpr sample which is the documentation and sample for TSslWSocketThrdServer, but does not support SSL. Angus