Angus Robertson

In which component are you seeing the access violation, IcsStrListToWireFmt should only be called in SSL servers. 

Angus

Don 't do much testing with Win64, unusual to find problems, particularly with a simple loop.  But that function is only used in one place in ICS, to decode the ALPN response during an SSL handshake, so the buffer is within the OpenSSL DLL, maybe there is an issue with the buffer alignment or something?  No ICS applications currently use the ALPN response, so I'll hide the exception as a quick solution. 

Is this problem with all SSL web sites or just one in particular, which is perhaps returning invalid data in the SSL handshake?  A typical ALPN response is just 'http/1.1, h2' so say that HTTP/2 is supported. 

Angus

Don't know anything about DEVEXPRESS components, but from reading your comments you purchased an ActiveX component from them which was written in Delphi, but which they no longer support, and you have no source code for it, but do have a similar VCL component with source? 

Surely the fastest solution is try and buy the ActiveX source from DEVEXPRESS so you can bring it up to date?  Perhaps even off to let them sell your new version for others in the same situation.  Recreating it 100% accurately is not a trivial task.  

Windows 10 generally retains compatibility with all earlier technologies, it's rare for Microsoft to obsolete APIs and stuff, but security often gets harder to implements, particularly because ActiveX was a horrible risk in MSIE. 

I wrote a non-visual COM object 10 years ago with a type library, for use with ASP web pages,  which was relatively easy, but a visual grid will be more complex.

Angus

Or a less specific topic to cover HTML components from other developers as well, such as https://github.com/BerndGabriel/HtmlViewer

Angus

If you use the component in the recommended way as I said earlier, it works fine, and does for many others.  There are no plans to change the internals of THttpCli, unless necessary to fix some horrible issue, it's been fine for 20 years.

Angus

The ICSLogger is designed for our internal debugging of the components, and needs extra information added to be useful for application level debugging, such as when you start a request, and the events called. It does not log the NTLM requests,. which is where your statuscode is coming from, not the real request.  Beware we don't often test NTLM since it so rarely used on the public internet.  

Angus

Not looked at the code, but generally status=0 means an internal error or something unrelated to the HTTP protocol like SSL or disk I/O, hopefully your application logs all the protocol commands and responses which will help indicate at what point the error happened.  If you use TSslHttpRest instead of THttpCli, logging is built-in.   

I would not rely on the component internal state completely, you should not start a new request until after OnRequestDone has been called, ie post a message from that event that triggers the next request in the queue,  

Angus

If you set the properties separately, don't use ProxyURL, that override them all.  It was add added to avoid needing to save four or five different properties, and pass them up from derived components.

Angus

The proxy sample handles non-SSL as well.  the four properties Proxy, ProxyPort, ProxyUsername and ProxyPassword are all set by ProxyURL, so don't use them together.  Never used ProxyConnection.  

Angus

ICS includes an HTTP proxy component and sample project which you can build and run locally for testing, OverbyteIcsProxySslServer.dpr. 

Assuming you are using V8.62 or later, there is a new property ProxyURL property which combines four proxy properties as a URL for simplicity, ie http://[user[:password]@]host:port.  You still need to set ProxyAuth if that is needed.  A proxy listens on a specific IP address and port, and then forwards traffic to the original URL.  For testing, you can set 127.0.0.1 and port 81, and set-up the proxy to listen on the same, so ProxyURL would be http://127.0.0.1:81. 

Angus

Did you get anywhere using the Google RSA-PSS private key?  I've made IcsJoseFindAlg recognise them OK, but then hit a problem in IcsJoseJWKPubKey because the OpenSSL RSA functions don't seem to work on RSA-PSS keys so I can not read the exponents needed, I think this was why I have up testing RSA-PSS 18 months ago hoping OpenSSL would fix this, but not yet.  There are possible workarounds.

Angus

Update: OpenSSL changed the RSA functions to recognise RSA-PSS keys in October, but not had a new release of 1.1.1 since, so we need to be patient and it will work soon.

No, the ICS HTTP server is independent of the Windows http.sys API, While a server running at kernel level is potentially more efficient than one at application level, all the REST and authentication stuff would still be at application level.  You are also restricted to server facilities Microsoft chooses to offer, which are present means no TLS/1,3 or modern ciphers, for instance.  So really only an advantage for very heavy load servers.

Angus

Not much error handling for opening the file, it might not exist or be protected, or whether you read it correctly, I set all the output parameters for PKCS12_parse to nil before calling it, unless this is a very old Delphi your password is not AnsiString, just a few things to try, OpenSSL error handling might give you some ideas.  Your last line does not work with any newer versions of OpenSSL, and 1.0.2 is out of support in four weeks.

ICS has a TX509 certificate class that does all this for you, including getting all certificate fields, and another that renews it automatically before expiry.  You can use these with internet libraries. 

Angus

ICS is a project developed by volunteers and offered free of charge to the community.  Volunteers come and go, and currently there are none helping with C++ and MacOS, so our level of support depends on reports from end users, and we try to react.  We spend our time developing for platforms used by the majority of Delphi users, if those on other platforms don't help, they should not expect support.

V8.60 earlier this year added a lot of new components and it seems no MacOS user has tried to build this, thus the errors were not found.  So download V8.59 or earlier which should be okay for MacOS. The last bug specifically fixed against MacOS was in V8.52.   

I will fix or workaround GetComputerNameW and GetThreadLocale,  etc etc etc does not really help, there are not many new APIs.  I really don't have the time spend hours on this stuff, I just need to be told what to fix.

Angus

Did you try removing the offending Windows only SystemTime functions?  Did ICS then build okay on MacOs64?  I've corrected that problem in master, but need to know if other MacOS errors then occur.

Angus

It was concept code, how to get the port of an open socket. and can be simplified somewhat to:

BindIpPortStr := Socket.GetXPort;

since it's a built function that the FTP client seems not to use.  In this case Socket is your TSocketServer component, provided you are not using IcsHosts. 

The IcsHosts implementation up to V8.63 does not allow a zero port, since that means there should be an SSL port specified instead, each IcsHost is designed to listen on two ports at once.  But this was a bad design, so I'll change it for the next release so that both ports being zero uses a non-SSL random port.  I'll also return the random port allocated, somewhere. 

Angus

The port number property does not change if you specify zero, it remains zero.  You need to find the port number allocated by windows:

    saddr        : TSockAddrIn6;
    saddrlen     : Integer;
 

           { Get the port number as assigned by Windows }
            saddrLen  := SizeOf(saddr);
            ListenSocket.GetSockName(PSockAddrIn(@saddr)^, saddrLen);
            DataPort  := WSocket_ntohs(saddr.sin6_port);
 

This should work with the listen socket in SocketServer, but other things that need the port won't know about it. And not tested for zero.

Angus

TWSocket allows the port to set to zero and listen called with a random port allocated by Windows, this is used by the ICS FTP client for the PORT command.  

Not sure about TWSocketServer, never tried it but it's rather more complex listening on multiple ports so ignoring zero is quite possible, TWSocketServer is certainly not designed to listen on a random port.   Again the FTP client has code to allocated ports sequentially from a specific range, trying the next if in use, so you could borrow that.

Angus

A RSA-PSS key should be PS256 not RS256, but this is really down the other end, whether it accepts both type of RSA keys for RS256.  Easy to relax our code, if the server is also relaxed.  I'll look at this stuff when I do SASL for gmail. 

Angus

And what happens when you build for MacOS?  I think you will get the same errors.  It seems GetSystemTime is Windows only so need some conditional code there, guess no-one has tried to build for MacOS for a few months since that was added.  You can try making those two UTC function dummies removing the real code, I think they are only used for the time client and server.  Sorry, I won't be looking at this immediately.

Angus

I did finally get a Twitter developer account a while back, but always seem to be fire fighting something else when I try to get back to OAUth1.

Angus

I was testing Google Accounts last week, it's OAuth2 implementation is unusual but now supported by ICS, did not get as far as trying a service account.  But if Google is returning RSA-PSS keys, we need to support them, so on my list.

Angus

Quite happy to make ICS compatible with MacOS64, but you need to tell me what errors appear in which units, I don't have any Apple hardware to test.  Or is this a DEFINE issue, with MACOS or POSIX not being declared on MacOS64?

Angus

This list is possible additions to ICS, new protocols and functionality, none of which is guaranteed...  Open to suggestions for other possible protocol additions or improvements.  Personally, I'm unlikely to look at any of this stuff for several months, unless my company has an urgent need for something new.  But if several other users are all looking for the same thing, I can help co-ordinating improvements.

Protocol: STUN client and server
Why: Session Traversal Utilities for NAT allows finding a public IP address while behind a NAT router, by contacting a STUN server.  Used by public servers and clients that need to tell other applications how to contact them.  Also some client protocols like a host name, like SMTP.
Difficulty: low, simple protocol, easy to implement.
Benefits: medium, saves configuring the IP manually.

Protocol: RDAP Client
Why: Registration Data Access Protocol is the replacement for the Whois protocol, using HTTPS REST and Json protocols. Both domains and IP addresses.
Difficulty: low, simple protocol, easy to implement.
Benefits: low, Whois is heavily censored now.

Protocol: Roughtime client and server
Why: replacement for NTP and SNTP network time protocols (from Google), with security.
Difficulty: low, simple protocol.
Benefits: low, usually get time from Windows.

Protocol: HTTP/2 for HTTP client and server
Why: More efficient version of HTTP/1.1, particularly for web pages with dozens of elements, compresses headers.
Difficult: moderate, extra DLL, messy, lots to change.
Benefits: low, ICS is rarely used to download complex web pages, perhaps
more important for the HTTP server.

Protocol: SASL for SMPT and POP3 clients
Why: Simple Authorisation and Security Layer adds OAuth2 for SMTP and POP3, safer than clear authentication.
Difficulty: low, OAuth2 already done.
Benefits: high, where the email provider requires it.

Protocol: OAuth1 for Twitter
Why: Twitter uses OAuth1 rather than the easier and more recent OAuth2 almost everyone else uses.
Difficulty: low, uses HMAC which is done already.
Benefits: high, if you want to send tweets.

Protocol: DNS over HTTPS (DOH)
Why: secure DNS can not be intercepted and modified.
Difficulty: ICS already has a TDnsQueryHttps component and sample, but causing it to be used by TWSocket and other components at low level could get messy and link in all the REST and Json units.
Benefits: low, Microsoft is threatening to support DOH, probably only
Windows 10/2019.

Protocol: MQTT
Why: MQ Telemetry Transport is used to send messages between devices, including IoT.
Difficulty: ICS MQTT project n GutHub, needs integration.
Benefits: high, if you need the protocol.

Protocol: Websockets server
Why: A full duplex version of HTTP often used for server push to dynamically update web pages.
Difficulty: ICS websockets project in GutHub, needs update and integration.
Benefits: high, if you need the protocol.

Protocol: SChannel SSL/TLS Support
Why: avoid distributing OpenSSL DLLs by using SSL/TLS protocol APIs built into Windows. Downside is Microsoft often takes years to support new protocols and often only in the latest operating systems

Difficulty: high, needs to be done at the lowest levels, risks adding bugs for OpenSSL if both supported, need to replace a lot of OpenSSL encryption APIs with Windows APIs, and certificate APIs.
Benefits: low, unless you really hate DLLs.

Protocol: POP3 Server, IMAP Client and Server
Why: because these are missing and we all use email.
Difficulty: moderate, lot of new new code.
Benefits: high, if you need them.

Platform: Better C++ and MacOS Support
Why: we don't do much testing on C++ and MacOS due to lack of volunteers to do this regularly.  We lack samples for C++ and MacOS.
Difficulty: high, users want someone else to do the work.
Benefits: high, for C++ and MacOS users.

Platform: Support for mobile apps and Linux
Why: more platforms.
Difficulty: very high, probably at least one man year effort, maybe more.
Benefits: high, for mobile apps and Linux.
 

Angus

Which previous version did you successfully build under MacOs64?  The announcement said MacOS and Win64, not MacOs64. 

Angus