Angus Robertson

KeepAliveTimeSec is an idle timeout, from when the last data was received or sent on the client connection. It is part of http/1.1 that leaves the connection open after a request, waiting for further requests and defaults to 10 seconds in all ICS web servers, XferSecs is five minutes. It's not changed in years, except to support Int64 ticks. Is this Windows Server or MacOS? Why FMX for Windows? ICS v9 has the first new FMX samples in 10 years, I suggest you build the FMX sample IcsSslMultiWebServ and see if you can reproduce the problem, if so I'll look into it. Angus

When you say 'big performance regressions', if I read your article correctly you mean some cryptographic functions are slower in new versions, due to the provider layer that hides internal structures from being damaged by applications and allows flexibility for developers. The question is how many times a second are those operations performed in a typical web client or server, so what is the actual penalty? Or is it once or twice a connection, so microseconds? Angus

My public web servers have the line: Client.KeepAliveTimeSec := 120 ; // 13 July 2009 increase session timeout and serve thousands of users a day without halting since that line was added. I don't use KeepAliveTimeXferSec which is only effective during requests, Both versions close the client, not the server. Which ICS version are you using. Angus

The OverbyteIcsOAuthFormVcl unit is very much optional, it is new in this release and not even built for Delphi 7 since it does not have the browser windows. But a couple of samples will complain. SHDocVW is not in any ICS packages. We only provide very limited for support for very old compilers, except Delphi 2007 which is widely used, my XE license has expired so I can not use it, only some of the later XE versions. Angus

Thanks, I'll look at this when I get back to ICS next week. Note, there is an ICS forum for future support, but you don't need to repost this topic. Angus

It is no secret that many or most components developers are beta testers under NDA, how else are all their components ready for each new release, or in the olde days on the component companion CD included with the final release. This benefits everyone involved, because new versions of Delphi can be used for old projects immediately, rather than waiting weeks for developers to buy the new version, etc. What has changed in recent years is beta testing being offered openly for paying customers, rather than by invitation only, and blogging about the next release, so it is now all more obvious. Angus

I assume you mean the files OverbyteIcsDXeRun.dpr and OverbyteIcsDXeRun.dproj? SHDocVW is not in either of those files so your compiler must have added it. TWebBrowse did keep changing in early versions of Delphi, ICS has several painful conditionals to try and make it work. You can try removing OverbyteIcsOAuthFormVcl, and refeences to it, that might help. Angus

Component libraries can be published that have been tested on beta versions, provided they don't reveal any features of the beta or use any new features. The latest ICS v9 release has packages for Delphi 12, but did not need any changes for D12 other than VER360. Several other libraries are also available from GetIt for the beta. Angus

The X509PublicKey property returns a PEVP_PKEY pointer which can be passed for processing to numerous OpenSSL functions, PEM_write_bio_PUBKEY might do what you need, it is used in the PublicKeySaveToText method, but that needs a private key which you don't have. PEM_write_bio_PUBKEY returns base64 ASCII text, the same as you'd find in a PEM file for a public or private key, which is probably what you have already, it probably has the top and tail headers... Angus

ICS exposes the two hashes for the certificate, which are small and easy to check, but this would only work if there is only a single self signed certificate on the network, if there are more than one all signed by the same private key, like a CA, then checking the public key is the only solution. But no-one else has ever needed it. It is only a couple of lines of code using OpenSSL functions, but not this week. Angus

Sorry, no ICS applications need to use or display a raw public key, so there are no methods available to get it as a string. The TX509Base property X509PublicKey returns a pointer to the internal OpenSSL key, but there are no ICS functions to convert this to a string. There are some Jose functions for JSON Web Keys but these need private keys, not public. You can see the use of GetPKeyRawText in the OverbyteIcsPemtool sample, it prints all fields from a certificate, including the public key in hex, but you would have to parse the result to get the hex only. As I said before, applications normally check certificates, not keys. Angus

Checking the public key is exactly how chain verification works. Your self signed certificates should really be signed by your own certificate authority, or ideally an intermediate issued by your own CA. You distribute the CA certificate to your PCs and install it in the Windows and/or PEM store, and all normal certificate chain validation just works, for any application. The OverbyteIcsX509CertsTst sample will create certificates signed by your own CA, or intermediate, I use them for testing on my LAN. If you want to check the server certificate chain yourself, use the OnSslHandshakeDone event. Angus

The OverbyteIcsHttpRestTst sample illustrates SSL/TLS certificate validation, if that is what you mean by key data. You don't have to write code or make decisions, it's all handled automatically by ICS, if you set property CertVerMethod to CertVerBundle or CertVerWinStore. ICS has built in root bundles for certificate validation. Property SslReportChain will report the chain for your log, while SslRevocation will cause an OCSP server to be checked as well. Angus

The comment you quote from the source code is from four years ago, and relates to code contributed and tested by an ICS user OAS. I can not test it since I don't have an NT domain. I can only suggest you search that unit and OverbyteIcsSspi and OverbyteIcsNtlSSp for comments by OAS who made the changes. Angus

If I interpret your short hand correctly, you now have one global SslContext and a second one in the thread. Unless you actually initialise the global SslContext or attach it to a component and make an SSL request, it will not load the OpenSSL DLLs, so your wasteful problem of loading and unloading the DLLs several times a minute will remain. Please read my previous messages where I have explained how to do this properly, I'm not going to keep repeating myself. Angus

There would only have been C++ fixes if someone else found a problem and told me, I don't write or test C++. You should try ICS v9 which is released now and report any issues, ideally with fixes. Angus

The usual way to ensure OpenSSL is only loaded once is to drop an SslContext on the form, or create it once when the program starts. Ideally you initialise it once as well, since that is when the DLLs are loaded, and check to see if the DLLs are actually available to report errors before requests start. The high level components and servers have multiple SslContexts so in that case you call OverbyteIcsWSocket.LoadSsl when the form is created, and OverbyteIcsWSocket.UnLoadSsl; when it's destroyed, as illustrated in numerous samples. You normally set several global variables before calling LoadSsl depending on whether you want old or new OpenSSL versions to be loaded, or from a specific directory, whether you need the legacy DLL, or checking the code signing signature for malware, again in all those samples. Angus

I never said MultiThreaded would solve your problem, I said it was a mis-use of ICS for threads not to use it, it might work in simple cases, but not in most applications. I've already answered most of your other questions with previous comments. ICS how no knowledge of threads. It does reference count loading OpenSSL, but that only works if you free components correctly, so OpenSSL also gets unloaded correctly. Clue: what happens with errors when there is a large amount of code in a try/finally/end, and when you don't close connections first. Angus

IOCP may have some benefits with threaded applications, but those are very rare with ICS. I did some FTP testing a few years ago, ICS was opening over 100 non-SSL connections each second, slower with SSL due to all the negotiation that goes on and IOCP would be irrelevant there. I thought IOCP with TCP was mainly for speeding up transmission, sending larger blocks or files or something. But ICS is quite fast already, I transfer files using ICS FTP between my two public hosted servers in different data centres each night, and file transfer is usually 300 to 400 Mbit/s, less than the gig ethernet port speed, but respectable for the public internet, via routers and firewalls. Angus

Why would you need IOCP with ICS? Angus

Surely those headers are sent with the WebSocket Upgrade request? Why send them again perhaps each second on the same TCP channel, what would the server do with them? Angus

I guess the point is the WebSockets protocol is supposed to be lightweight, and not cluttered with unnecessary headers. Why are you not sending the header information as WebSocket data packets. A configurable option would be needed to bypass that clean-up code. I'll put it on the list, but taking a break from ICS for a couple of weeks. Angus

ICS has various components for SMTP mail, all with the latest SSL/TLS., free from https://wiki.overbyte.eu/wiki/index.php/ICS_Download Look at the OverbyteIcsMailQuTst sample which can now be downloaded pre-built. Angus

Full release notes for V9.0 will follow next week, meanwhile these are the main changes: New samples Samples/Delphi/SslInternet/OverbyteIcsSnippets.dpr - Small samples of codes for FTP, HTTP, sockets and email. Samples/Delphi/OtherDemos/OverbyteIcsNetMon.dpr - Internet Packet Monitoring Components, display packets and traffic using Npcap and raw sockets. Samples/Delphi/OtherDemos/OverbyteIcsNetTools.dpr - Network Tools Demo, uses all the main IP Helper functions, also TTIcsNeighbDevices, TIcsDomainNameCache, IcsDnsQueuy, TDnsQueryHttps, TIcsWhoisCli, TIcsIpChanges, TPing and TPingThread. Samples/Delphi/PlatformDemos/IcsHttpRestTstFmx.dproj - FMX HTTPS REST and OAuth, Send SMS and DNS over HTTPS functions demo. Samples/Delphi/PlatformDemos/IcsSslMultiWebServ.dproj - FMX Advanced multi host web server demo. Samples/Delphi/SslInternet/OverbyteIcsMQTTst.dpr - MQ Telemetry Transport message queuing service. Note this sample needs the VirtualTree component to be installed. Major sample updates for new components Samples/Delphi/SslInternet/OverbyteIcsHttpRestTst1.dpr - Uses TSslWebSocketCli for WebSocket Client, New embedded TOAuthLoginForm window using TOAuthBrowser for OAuth2 logins. Select client SSL certificate from the Windows Certificate Store. Samples/Delphi/SslInternet/OverbyteIcsSslMultiWebServ.dpr, OverbyteIcsDDWebService.dpr - Uses THttpWSSrvConn for WebSocket Server. IcsHosts can use server SSL certificate from the Windows Certificate Store. IcsHosts can now request a SSL certificate from the remote client. WebSocket server support. Uses TIcsDomainNameCache for multiple reverse DNS lookups. Samples/Delphi/SslInternet/OverbyteIcsPemTool.dpr - Can now export an SSL certificate from the Windows Certificate Store with its private key. Samples/delphi/OtherDemos/OverbyteIcsBatchDnsLookup.dpr - Uses TIcsDomainNameCache for multiple lookups. Samples/Delphi/SslInternet/OverbyteIcsSslMailSnd.dpr, OverbyteIcsSslMailRcv.dpr, OverbyteIcsMailQuTst.dpr - New embedded TOAuthLoginForm window using TOAuthBrowser for OAuth2 logins. Samples/delphi/OtherDemos/OverbyteIcsNsLookup.dpr - Uses single or multiple DNS servers, including built-in list of public servers, also sync requests. New Components TIcsDomainNameCache and TIcsDomNameCacheHttps - Cache forward and reverse DNS lookup requests, using several methods. TIcsMonSocket - Internet monitoring using raw sockets. TIcsMonPcap - Internet monitoring using Npcap NDIS driver. TIcsIpChanges - Monitors IP address changes dynamically. TIcsNeighbDevices - Builds historic LAN MAC device and IPv4 and IPv6 address table using ARP, neighbourhood and IP range scanning with reverse host lookup. TOAuthBrowser - OAuth authentication browser window VCL/FMX form. TSslWebSocketCli - WebSocket client protocol. TIcsMQTTServer and TIcsMQTTClient - MQ Telemetry Transport message queuing service, client and server. Major Component Upgrades TDnsQuery - Add synchronous methods and more response properties. Check multiple DNS server hosts including public DNS lists. TSslWSocketServer - IcsHosts can use server SSL certificate from the Windows Certificate Store. IcsHosts can now request a SSL certificate from the remote client. TIcsFtpMulti - Send NOOP command periodically during multi hour transfers so connections are not closed accidentally. New classes and Functions THttpWSSrvConn - WebSocket server protocol. Internet Helper Functions - Unit OverbyteIcsIpHlpApi.pas includes IpHlpConnsTable, IpHlpAdaptersInfo, IpHlpAdaptersAddr, IpHlpIpAddrTable, IpHlpIpNeighbTable, IpHlpIPForwardTable, IpHlpIpPathTable, IpHlpGetDnsServers, IpHlpIfTable2, IpHlpIPStatistics, IpHlpUDPStatistics and many other functions. TIcsMonFilterClass - Filter network traffic on protocols or IP addresses. TIcsTrafficClass - Maintains network traffic statistics by protocols and IP addresses. Angus

Thanks, fixed. Angus