Angus Robertson

My mail server does not block domains, but IP ranges used by spammers and hackers, about 800 at the moment, three emails this morning trying to get my email account password, so two more hosting companies in Italy and South Africa now blocked, third was from a Google user account, sad that Google allows hackers to send faked emails. If it's my server blocking you, which is rare, please send the log as a message via this platform. Angus

All the old references to specific Delphi versions like 110, 104, 120 etc should be replaced by New. I thought I'd done that, but seems I missed a few, and since I don't do C++ it needs to be tested by someone else. Angus

I've made a change to TSslHttpRest adding a new way for applications to check SSL certificate chains themselves, ignoring OpenSSL bundle checks, usually for self signed private certificates. if CertVerMethod = CertVerOwnEvent, during OnSslHandshakeDone the component calls a new event OnSslCertVerifyEvent where the application can check the chain and change the verify result appropriately. Maybe checking certificate serials, names or public key. Not tested yet, need to update the sample with the new event and some sensible code to check a self signed certificate, not until next week. Supportng the new OpenSSL for raw public keys will take longer, need to do the server stuff first so I can test clients. Angus

Error 451 is not something I often see, not using my servers. So please email logs generated by the component showing all FTP commands in a session, to the email you'll find in the source. I'd also recommend you test against other FTP servers to see if it's a client or server problem, you can upload to ics.ftptest.org which is the ICS FTP server, but you'll need a login I'll send you when I have your email address. Angus

Effectively, when validating the chain, OpenSSL and ICS are checking the chain is signed by a public key in a certificate in the store, you are just trying to skip the certificate overhead. I've found the new OpenSSL APIs to make the server use a raw public key instead of a certificate, and likewise the client to check it, now need the time to test it all. Angus

The best way to handle self signed certificates is the way ICS does it, the server components use an ICS intermediate certificate to create CA signed certificate if a real one is missing. ICS includes two intermediates signed by the ICS root, a short 200 day intermediate version for which the private key is distributed, and a two year one for our use only. You can use these or create your own versions with ICS samples or components, create your own CA signed server certificates, or if using our servers let it happen automatically, and ensure your clients have the root certificate in the store, only that certificate no others are necessary. The ICS root gets loaded automatically for all ICS applications. If you want to use real self signed certificates, TX509Base has a method X509PubKeyTB that gives you the public key, save it on the server and distribute to your clients in some way (instead of a root), extract the public key from the certificate in the OnHandshakeDone event and compare it with your saved version. The only catch when using the high level components with built in chain checking is there is no chain if you disable checking, but if you enable it, our checking can not be skipped. I'll need to fix this somehow when I add public key checking. Angus

I do have a couple of intermittent FTP bugs on my pending fix list, but since my downloads and uploads always work after retries, not got around to investigating them yet. The github report relates to the socket being closed too early, ICS normally waits until all data is processed before closing a socket using the CloseDelayed method, But I'll look at the code, next week. Angus

You are trying to do something unusual, I'll need to look at the code before I can make any suggestions, may be a day or two. This has been done before for light weight clients. Also OpenSSL now supports RFC7250 to check certificates by raw public key, which I plan to support. Angus

Essentially your code is ok, except the first parameter of AnswerStream is var Flags : THttpGetFlag; so can not be left blank. There is a an extra parameter to pass a last modified date which is friendly for so applications that do a HEAD first to check if the file is newer. You should free DocStream before creating it, and the server should free it when the request completes. Angus

CertVerifyMethod := CertVerNone is how you bypass internal chain verification. TSslWebSocketCli is derived from TSslHttpRest which has an internal TSslContext, and it's that component that does all the SSL stuff. You can use an external SslContext, see the notes in the rest component, for instance if you are using a hundred rest component instances to download lots of stuff in parallel. If you are using the real WebSocket protocol, it exchanges frames between client and server, and vice versa, and there are several methods WSSendText, WSSendBinary, WSSendFrame, etc, with the OnWSFrameRcvd event being received frames, which you need to check the data type and then process. IsWSConnected does as it says. If you want to send raw data without using WS frames, you are using the wrong component. Angus ,

As I said before, the option 'no temp file for xfers' magftpNoTmpFile will avoid the rename issue. you tried that and it did not fix the missing file name issue. The component will zip files before upload, old versions used VclZip, new versions on new versions of Delphi use Delphi native zip. Angus

That non-SSL sample is no longer part of the main distribution, the SSL version was last updated two weeks ago in SVN. But the FTP commands have not changed in many years. Angus

I've tested the FtpUpOneFile function against my FTP server, and it works are expected. The only explanation for the missing file name in the logs is the parameter RemTarFile not being passed to the function, but unfortunately it is not logged other than when being used for FTP commands which all show blank in your log. So the only way to further diagnose the problem would be to add logging with RemTarFile at the top of FtpUpOneFile and FtpCheckFile. No point in my doing it in the current version since you are using an older version. Angus

The purpose of the Websocket is to exchange data with a server, so it opens a connection and does a GET to the path you specify, just as HTTP does, if you don't want that it's not Websockets. I suggest you try using the OverbyteIcsHttpRestTst sample I mentioned, if that does not work you would appear to be using a custom protocol. In a console application, you need to call the message pump after every line that accesses IP functions, ICS is fully event driven. The sample I mentioned makes a single HTTP command, waitis for it to finish and exits. If you are making a series of commands, the console applications gets more complicated. Angus

V8.01 is 14 years old, unsupported. Angus

Thanks for the comments, not all features get fully tested provided they don't break anything, I'll see what I can do improve matters, but it may be a couple of weeks. Angus

Don't believe it's a server issue, nor was there a timeout issue, it failed due to the client sending too many bad commands without the file name as an argument. You can always try uploading to my public FTP servers which are ICS based, the Snippets sample does FTP uploads and downloads, but you'll need a login for uploads which I'll email. If you do upload something, let me know the time so that I can check the server log, there is a lot of traffic each day. But I expect it to show the same issue. To try the latest ICS version, you can download the compiled Xfer sample, https://wiki.overbyte.eu/arch/icsdemos-clients.zip Angus

If you are using IcsHosts, you can not set any SslContext options, they are set by the component according to the SslSrvSecurity level you set, which uses internal cipher lists with Mozilla recommended cipher suites according to how much backward compatibility you want with older clients on ancient versions of Windows of Android. For TLS/1.3, it always adds the sslCipherTLS13 constant ciphers, which include CHACHA20. So perhaps the answer to your problem is one or more SslSrvSecurity levels that only have AES ciphers? Meanwhile, you could edit that constant to remove the first cipher in the list to fix your problem. Beware the TLS/1.2 ciphers also include CHACHA20. I probably have to revisit ciphers again since newer OpenSSL versions are adding newer ciphers, which we currently ignore. Angus

I've looked briefly at your log, and it seems a lot of early commands are failing trying to check if the file being uploaded already exists on the server (MLST, MDTM) due to the same missing file name as after upload. But the logging is not enough to show where the name is lost, it's a balance between debug bloat and being user friendly. The FtpUpOneFile function is not one I use in any of my current applications, only FtpUpload, although both use the same internal functions for all FTP commands. But it might be worth trying FTP Multi with source file name as your single file, and see if that works OK. I'll do some more testing of the single file transfer functions, but not today. The only thing to try would be a different file name, not an EXE, no idea how, but perhaps some AV software is corrupting these commands to stop you uploading EXE files, it's the kind of thing the end point protection gangs do in the name of protecting you from yourself. This is a very long shot, I hope! BTW, it not the FTP server, the commands don't reach it. Angus

There is an old console HTTP sample OverbyteIcsConHttp which is now in the archived samples download https://wiki.overbyte.eu/arch/arch-samples-V9.1.zip Essentially, it just has a message loop after the request, FHttpCli.MessageLoop; and the RequestDone event has FHttpCli.PostQuitMessage; which causes the loop to break and the application to die. Alternatively, you can call ProcessMessages in your own loop checking stuff until you are ready to break. I'll make sure the next release has at least one console SSL sample. BTW, you don't need an TSslContext with the new high level components, it's created internally. Angus

Can you please confirm your query only relates to the ICS SSL server, and not clients? From your comments, I assume you are not using IcsHosts to specify certificates and SSL parameters? Angus

There is a terminology issue here, Websocket is a specific protocol based on HTTP. TWSocket is the name of the base ICS component for all low level socket operations, and does not handle high level protocols like Websocket. ICS has a TSslWebSocketCli component, there is a simple function doWebSocketClick in the OverbyteIcsSnippets sample you can build and run which accesses one of my servers, or a full example in the OverbyteIcsHttpRestTst sample that should allow you to access your site. ICS has several console samples, but they are no longer in the main distribution not being needed very often, I'll have to look one out. Angus

Somehow, you are not passing a file name for the upload, or it being lost. But the component should stop you doing that... The reason may be in the log which should have the file name a few times already. Angus

As I requested earlier, please email the complete log for the transfer, selected excerpts are meaningless. But please disable low level diagnostics. The problem is clear from the log, renaming is failing due to no new file name, the log might show something earlier. You could set the option 'no temp file for xfers' which is a quick fix, that feature goes back to the days of dial-up modems where dropped internet connections were common. Angus

All newer high level ICS components have a logging event for display, progress and file logging, slightly different names in different components, the FTP one is CopyEvent as used in the Xfer sample. The FTP close problem I fixed was time related, a network appliance may close the FTP control channel during a long data transfer due to inactivity, so nothing happens after the data connection closes, The solution was to send periodic commands on the control channel, if the server accepts them. I had the problem after two hours. Angus