Angus Robertson

A message here on 12th August explained enough. To record the HTTP protocol, you add logging to component events, onCommand and onHeaderData in particular. But the better way is to change TSslHttpCli to TSslHttpRest which is effectively a higher level version that handles lots of stuff you have to plug into the former manually, like cookies, compression, SSL context, and logging. Look at http://wiki.overbyte.eu/wiki/index.php/FAQ_Using_TSslHttpRest Angus

I know HTTP proxy support works in V8.62, since I improved it a few months ago and use it regularly myself, to the ICS proxy server. I simplified the way it is configured so you can use a single URL rather than four separate fields, per the release notes I published here recently. So what proxy protocol are you using? Is this HTTP or HTTPS? Is there any redirection involved? Where are the protocol logs? Angus

The short term solution is to use OpenSSL 1.1.0 which should still work on Windows XP. We still provide new binaries for 1.1.0, but support ceases in a few months, so after that there are no more security fixes. At some point, ICS may stop supporting 1.0.2 and 1.1.0 to remove some conditional code, so you may need to then keep an old version of some ICS units as well, but probably not for at least another year. Indy users will also be hit by support for 1.0.2 being stopped, since I don't believe it yet supports 1.1.0 or 1.1.1. Angus

There are several different proxy options in different components. Angus

The OverbyteIcsHttpRestTst.dproj sample has a grid titled REST Parameters where you enter multiple Names and Values, select REST Content as Json, request type POST and enter your URL. If a Json response is returned, it is also displayed as a grid. The OverbyteIcsSslHttpRest.pas unit itself has several examples of setting RestParams using code, for making OAuth requests, DNS over HTTPS and SMS. and OverbyteIcsSslX509Certs.pas has numerous examples ordering SSL certificates. Beware the RestParams properties don't easily handle arrays, sometimes it's easier to build the parameters using a Jspn library, ICS includes SuperObject which works with all versions of Delphi, since Json was only added to the Delphi language in more recent versions. Angus

Thanks, your changes are in SVN. Angus

Your patch should be against the latest SVN version 1435, not a three month old version. Some of your changes in OverbyteIcsCB2009Run.cbproj have already been done. Angus

As I told you back in March, use the OverbyteIcsHttpRestTst.dproj sample which allows you to specify the Json parameters for your POST request. Angus

Probably not, since you asked the component to send from the end of the stream, not the beginning. you need Stream.Seek(0, soFromBeginning). If you are using TSslHttpRest most of thar code is not necessary since it builds Json for you. Angus

There is an example in the same unit, IcsJoseGetSig. There is also a sample project OverbyteIcsJoseTst.dpr that is used to test the various Jose functions, including hex, base64 and base64url encoding and decoding. But this unit needs the OpenSSL DLLs to be distributed with your application, as does any use of SSL or encryption in ICS. And you need to initialise OpenSSL before calling those functions, see the sample application. As others mentioned, there are Delphi only implementations of SHA256, but IcsHMACDigest more modern digests as well. and most REST applications need HTTPS.

BTW, the Json you receive from a websocket or by HTTP may not contain any line endings, so using ReceiveStr is a bad idea, it may not return complete lines, that might explain why your json was corrupted if there was no line ending at the end. That is why the HTTP component receives content into a stream, and that stream is passed to IcsHtmlToStr to convert into something Delphi can handle. Angus

For HTTP, the headers contain the content type and character set so it's easy to convert to a string, including entities like &pound;, look at TSslHttpRest.TriggerRequestDone2 in OverbyteIcsSslHttpRest.pas: // convert response to correct codepage, including entities if (Pos ('text/', FContentType) = 1) or (Pos ('json', FContentType) <> 0) or (Pos ('javascript', FContentType) <> 0) or (Pos ('xml', FContentType) <> 0) then begin FResponseRaw := IcsHtmlToStr(FResponseStream, FContentType, true); end; But I have no idea if websockets provides the content type, and you've not said if you are using Unicode. Angus

TSSLWsocket receives a stream of bytes, perhaps you are not correctly converting them into a string, might be UTF8, UTF16, ANSI or anything really. No idea what websockets specifies. Are you using a unicode or ANSI Delphi, the websockets project was ANSI only. I'm not aware of any up to date websocket libraries for ICS. Angus

This is avaiIable in the OverbyteIcsSslJose unit: { digests and hashes - note all digests are binary bytes in AnsiStrings } function IcsHMACDigest(const Data, Key: AnsiString; HashDigest: TEvpDigest = Digest_sha256): AnsiString; function IcsHMACDigestEx(const Data, Key: AnsiString; HashDigest: TEvpDigest = Digest_sha256): AnsiString; function IcsHMACDigestVerify(const Data, Key, OldDigest: AnsiString; HashDigest: TEvpDigest = Digest_sha256): Boolean; With various other functions used for Json Object Signing and Encryption, Angus

I don't see any problem with your proposed change, but don't see any great benefit either, the connection might close without OnDataAvailable being called so you still need processing in OnSessionClosed. Or am I missing something? Angus

Since I updated to Windows 10 1903 three weeks ago, some of my servers that use unusual ports have failed to start saying 'Another server is already listening on 192.168.1.124:1777' and similar. There does not seem to be a problem listening on common ports, 21, 25, 80, 443, etc, but not 2100, 4443 etc. I have always had Windows firewall disabled since this is a development PC and I have lots of internet applications being developed. All the firewall dialogs I can find are red warning firewall is disabled. But the Windows Defender Firewall service is running and can not be stopped, the options are greyed. I do actually have firewall rules set-up for some of the offending applications to listen on all ports. So is nanny Microsoft fighting my choice to not use a firewall and running it anyway? I could probably change the registry setting to stop it running, but Microsoft might check that and start it anyway. Anyone have silly problems on Windows 10 1903? Angus

Exactly like that thread from a couple of years ago. However the real issue here is it was not necessary until Windows 1903, and Windows internet applications that worked before may now fail. Angus

Just open a TCP connection to the server and you should get back a one line reply, that's all you need to see if it's online. But it might be off-line by the time you send email. And it might not accept email from you. The ICS library now has a mail queue component that allows multiple email servers to be specified, and all those servers are attempted if any fail. Angus

Resorted to regedit to disable the MpsSvc service, still started when set to manual, and now my applications are listening on all their strange ports again OK. So the Microsoft firewall is now running continually even when turned off according to all the dialogs. Will need to warn my customers. Angus

No, header lines will not be ignored or lost since they all end with CRLF, and a double CRLF at the end of the header. Headers are small, and need to parsed as strings. To receive a POST body, that function calls an event where you can accept data using Receive and write it to a stream if expecting a large upload or just process some Json or whatever. Angus

I've uploaded a 263x304 thumbnail photo to my profile, but I still see A appearing against my posts. The photo, actually a drawing, just appears cropped and massively stretched across the whole screen at the top. I'm missing something very obvious... Angus

Thanks both, the words 'profile photo' appearing anywhere on those pages would have helped, instead of expecting me to hover over icons. For those interested, the drawing was done by Steve Bell 40 years ago, he was political cartoonist for The Guardian newspaper for many years, and as a journalist at the time I commissioned an A3 cartoon of a pop concert not realising that my staff and I were going to appear in it. Angus

Why bother with torrents when you can download the official version from your local CDN node at 60 to 100Mbps, took a few minutes for 7GB. http://altd.embarcadero.com/download/radstudio/10.3/delphicbuilder10_3_2_496593b.iso MD5: 43FA7A32D9ECBB8CF14AF9519AFE066A Angus

I've just updated SVN and the overnight zip so that ICS will build properly again, tested against Delphi Rio 10.3 Update 2, aka 10.3.2, no changes were needed. There are some minor fixes from this forum, with a lot of other pending units waiting mostly relating to automatic ordering of Let's Encrypt SSL certificates, just got to fix one SSL bug in socket server. Angus

This change is now in SVN. Angus