Angus Robertson

Most people consider FTP dead now, replaced by HTTP POST/PUT, and never bothered to update their 20 or 25 year servers with more efficient commands introduced since then.  

MLSD actually has an RFC somewhere, while the data returned by LIST is undocumented, and does not always have a year in the date, it was historically a Unix directory listing, thus only line feeds.  FileZilla Server supports MLSD. 

Angus

When you improve the logging in the server to show errors and events, similarly to the sample you copied code from,. you'll get some debugging information.  See SslFtpServer1SslVerifyPeer,  SslFtpServer1SslHandshakeDone, 

And also fix the logging bug that is adding this path, not coming from the clients, HOST C:\TEMP\[127.0.0.1]

Angus

I always test the ICS FTP server with the ICS OverbyteIcsXferTst sample, then you just click a single button to perform a complete FTP session, with proper logging.  

Angus

What it should say is:

02:10:08  > AUTH TLS
02:10:08  < 234 Using authentication type TLS
02:10:08  Check for Old SSL Session
02:10:08  No Old SSL Session Cached
02:10:08  Starting SSL Session
02:10:08  Cache SSL Session: New
02:10:08  ! SSL Connected OK with TLSv1.2, cipher ECDHE-RSA-CHACHA20-POLY1305, key auth RSA, key exchange ECDH, encryption CHACHA20/POLY1305(256), message auth AEAD
 

But that depends on how many other events you are logging, you cam see in the sample where all this comes from (although this log is from a real server).  I suspect you are still testing with the OpenSSL client which to be honest is a waste of time, it does not understand the FTP protocol.   With TCP, it is rarely obvious which end caused disconnection. 

Angus

The FTP server sample does all the logging properly,SslFtpServer1AnswerToClient.

If you use IcsHosts in the latest server, you don't need an SslContext, all that is handled by the component. You set the certificates and an SSL security level and everything else is done automatically, including installing Let's Encrypt SSL certificates on a public server.  This all works now, just no new sample yet. 

Angus

Improving your logging will help, you did not report the response or errors to commands received which is essential to error tracing with bad clients. 

The latest ICS in SVN has FTP server improvements to simplify SSL configuration in the same way as the web and proxy servers two years ago using IcsHosts, but there is no sample to copy yet, you need to look at the multi-web server sample and see how that configures listeners and at http://wiki.overbyte.eu/wiki/index.php/FAQ_Using_IcsHosts. 

Angus.

The ICS TFtpClient component has no directory handling, there are no formal standards for directory formats with the LIST command,  you should use the MLSD command which is supported by all proper FTP servers and is standardised. Otherwise your application is responsible for the different directory listing formats when using TFtpClient.  The ICS FTP server has even more efficient directory listing commands that handle sub-directories.

But you are probably using the wrong component, 15 years ago I wrote a higher level ICS FTP client component which is now part of the main ICS distribution as TIcsFtpMulti which is described briefly at http://wiki.overbyte.eu/wiki/index.php/ICS_V8.60 with a new sample application OverbyteIcsXferTst.dpr.  This handles all the directory listing stuff for you, at least for most known common FTP servers, there are always developers with strange implementations. and automates uploading and downloading whole directories and sub-directories of files. 

Angus

I have no idea how the openssl client tool works or why you are using it to test FTP, but 'AUTH C:\TEMP\TLS' is not a valid command, it should be AUTH TLS'. You have not shown what smartftp sent.

Since the ICS sample works, you have missed something in your implementation but I can not debug your code. 

Angus

You don't log the IP address and port your server is using, although your client says port 5420 which is not an SSL FTP port, so the server does not attempt to negotiate SSL   Using non standard ports for SSL requires extra code.  V8.64 has a new Option ftpsAuthForceSsl that might do it simply.

Not sure why you are testing an FTP servers with a non-FTP client that does not send the correct protocol. 

Angus

Not sure what you are trying to achieve, but I suggest you get the original ICS sample working first, then work out what is different with your project. 

The IcsLogger is for our development purposes, not debugging applications.  Your code has lots of Logit commands, it's what those report that helps with debugging.

Angus

Quote

The one and only way to do this is how i described .. what i told you is best practice and how it is done, there is no other way 

So how do you suggest the password is entered for a background service application on a hosted server?   Using a token from an authentication server is fine, but how do you get it?

Angus

The code itself is probably OK if you copied it correctly. the issue is more likely all the settings and SSL/TLS certificates that are needed to make a secure server work, of which you show none.  

You could also have shown the protocol that you logged with all the errors.   What SSL certificate are you using, and what intermediate, since you are setting SslCaFile.  But servers don't use SslCAPath.

Angus

Updated OpenSSL to 1.1.1f, this reverts a change in 1.1.1e that caused problems in some applications, but I don't believe ICS was effected.

Angus

Encrypting a string or password is easy, harder is how you protect the key used for the encryption. 

Angus

The Internet Communications Suite (ICS) has a REST client component using Json and an application web server that sends whatever you give it.  There is an ICS topic here.  

But no SOAP protocol as such, Json seems to have overtaken SOAP as much easier to use.  ICS uses SuperObject for Json and reading XML, all Delphi 2007 compatible.  Adding a SOAP wrapper to your XML objects should not be too hard. 

Angus

Servers use SSL/TLS certificates, not usually clients.  Some errors on a certificate will prevent a server accepting an SSL/TLS connection, like an invalid private key. or cause the client problems like an expired certificate, but using self signed certificates is optional in the client, you decide whether to accept them or not.

Angus

Two new zips for Win32 and Win64 versions of OpenSSL 1.1.1e can now be downloadable from the Wiki at: http://wiki.overbyte.eu/wiki/index.php/ICS_Download or https://www.magsys.co.uk/delphi/magics.asp . The latest 1.1.1 DLLs are also included in the ICS distribution SVN and overnight zip.  This release includes one low priority security improvements and bug fixes, including one that allows IcsJoseJWKPubKey support RSA-PSS keys.

ICS applications require V8.57 or later to support OpenSSL 1.1.1e.

Changes in 1.1.1e may be found at https://www.openssl.org/news/openssl-1.1.1-notes.html 

Angus

There is nothing wrong with Delphi 7 for applications today, if anything they are less bloated than later compilers where EXEs just get bigger with no extra functionality, All my ICS development and commercial applications are Delphi 2007, but then tested and made compatible with the latest and greatest of the day.  Obviously cross platform development is another matter, but I don't do that.  And lots of new language features and library functions, which ICS seems to manage without, or has implemented it's own versions to be compatible across all compilers D7 up. 

For client/server, also look at the new ICS component TIcsIpStrmLog and sample OverbyteIcsIpStmLogTst, the component can be configured as TCP/UDP/Client/Server with SSL/TLS, and mainly just sends lines of text back and forward, with two methods to connect/listen and send and one event to receive, you can build client/server apps very quickly. 

Angus

Websockets are designed for use in browsers, as a more efficient duplex mechanism than Ajax, for interactive content like spell check and auto completion as you type.  

Don't see any purpose outside the browser environment where simple TCP client/server has been used for 30 years.  ICS has several sockets demos illustrating various client/server concepts. 

Angus

When I get feedback from people actually testing this initial implementation, I'll look at it again. 

Angus

I've updated the ICS distribution with the Websocket server implementation (HIXIE and HYBIE protocols) ported from phpws project, written by Stan Korotky <stasson@orc.ru> (eight years ago).

This demo may be tested against the  web page websocketclient.html which should be accessed from a server using the same IP address as this application, such as the ICS demo web server.  The web page uses the
WebSocket API with Javascript to send data to the server which is echoed back to the page.

Ideally, websockets should be integrated into the ICS web server so they can be used on the same ports, the request that starts them up is HTTP, and I'll do that is anyone needs it. 

Please note this is an experimental server websocket implementation, that needs a lot more testing in case the protocol has changed in the last eight years.  But it works with the latest version of Firefox as a client.

A websockets client component is more work, but using most of the same code as the server since it's a duplex protocol. 

Angus

Websockets is a client server protocol, like HTTP, except once the connection is open duplex packets can be sent back and forward.  So that packet stuff is common to client and server, and exists in the old code using ICS.  The sample application is just a server and a JavaScript client web page.  But creating an ICS client should not be hard, uses HTTP and the same functions as the server.

Angus

I will do a quick evaluation of building that old code with the latest ICS over the next few days, but I won't do anything further unless someone else sets up a websockets test environment to test it all, that often takes as long as the coding, a server and client that the ICS components can be separately tested against.

Angus

I added your change to my copy, not in SVN yet.

Angus

Thanks, now fixed.

Angus