FTPServer Beginner

[Mark Williams 13]

I am looking for some general guidance on how to go about setting up an FTP server for the first time. I have opted for TIDFTPServer for this purpose and have been through the help files and searched Google, but I have come up with very limited information.

 

I don't want to launch into this in my usual bull in a china shop manner only to find later that I have gone about it in the least appropriate way. So apologies in advance for the broad nature of this topic.

 

In essence my requirements are for a basic FTP Server. However, I want to try and strictly control what users can upload and, in particular, download.

 

My fledgling thoughts:

 

Uploads

The user requests permission for upload via my webserver and if authenticated is issued with a unique token to be stored in the database along with an expiry time which the user will need to renew periodically via the webserver and an identifier to identify which folder(s) the user is allowed to upload to.

 

The FTP client would submit the token to the FTP server which would retrieve the token, expiry  time and folder identity prior to permitting any upload request. When the client has finished it will request the deletion of the token.

 

There are likely often to be large numbers of files to be uploaded so preferably the FTP Server will be able to store the token, expiry time etc rather than querying the database for every upload. The FTP client would submit the token on every upload request.

 

The FTP Server would check periodically whether the token has been removed.

 

Downloads

There may be a requirement for download of a large batch of files. So similar approach to the above. I already keep a record of every file in a database so each has a unique id and there is another table which records user rights to access a given file.

 

Before submitting download request user applies for token via webserver and at the same time submits a list of ids of those file he intends to download. Webserver checks which ones user has access to and populates a table with the file ids ("permitted_files").

 

On receiving the retrieve request the FTP does the same as for upload, but additionally queries the "permitted_files" table to see which files the user is permitted to download. 

 

Questions

1. Is this broadly a sensible approach?
2. If not, how better to go about this?
3. Assuming it is sensible, what do I need to implement this within the FTPServer component. My best guess at the moment is that it would be handled via CommandHandlers. 
4. No GUI interface is required and I don't really need a command line so how best to implement this on the Server: as a Windows service? 
5. As the FTP Server may be receiving numerous requests at the same time how best to manage the data relative to specific requests (ie tokens, permitted file ids etc).

 

[Sherlock 627]

Will this server be open to the public? If so, please consider a finished, polished and most importantly relatively safe commercial/OS product. Though your token solution sounds good, it may not be once you have implemented it. Exposing yourself or maybe worse your customers like this should be given a lot of thought - and know how. First off, you should use FTPS or SFTP, not plain old FTP. If I had to, and I mean really head to, I would use FileZilla and for each user their own folders, without possibility to get out of that (resulting most likely in a Linux file system). Then into every folder a sym link (or whatever the're called)  to the files they may retrieve.

[Guest]

1 hour ago, Mark Williams said:

Is this broadly a sensible approach?

Yes.

1 hour ago, Mark Williams said:

If not, how better to go about this?

I would suggest to go with HTTP instead of FTP for some reason:

1) HTTP is router and firewalls friendly, you will ask any user of yours to open ports or configure port forwarding, hustle free!

2) HTTP or HTTPS is easier/simpler for you to manage on server side for both your FileCenter along with the web application ( your webserver), even you can use the same/one application to manage all of that.

3) Is HTTP slower than FTP ( or is HTTPS slower than FTPS/SFTP ) . NO 

4) Does FTP provide simpler way to manage file permission, NO, same for both approach's, with small differences.

5) with FTP or HTTP you really should consider using encryption ( TLS or not)

 

1 hour ago, Mark Williams said:

Assuming it is sensible, what do I need to implement this within the FTPServer component. My best guess at the moment is that it would be handled via CommandHandlers. 

I am not well versed in Indy, in fact last time i used it was around 10 years ago, may be others can provide insight on that.

1 hour ago, Mark Williams said:

No GUI interface is required and I don't really need a command line so how best to implement this on the Server: as a Windows service? 

Sure, Windows service is always the best, but here i want you to read the down notes.

 

1 hour ago, Mark Williams said:

As the FTP Server may be receiving numerous requests at the same time how best to manage the data relative to specific requests (ie tokens, permitted file ids etc).

In case you want a separated FTP server, i can't provide much help here , except i used and i am using FileZilla as Sherlock suggest for simple FTP server usage, Only here i may suggest to use the portable version from XAMPP, it is simple folder copy it where ever you want and will work.

 

 

Now to details in case you considered HTTP with RTC :

I would suggest RealThinClient (RTC SDK) https://rtc.teppi.net/ ,it is one of the most prized and powerful library i ever had, it is not free but the price is relative to its usefulness, all what you mentioned above can be done with RTC in few hour when you are well familiar with RTC components, one problem though to keep in mind, the author is giving up on most of its functionality and deprecating many features.

In case you considered it, i suggest to go ask the author before buying, i put this library in must have column for everybody.

Some advantages/notes from using it:

1) Same code can be a stand alone server, windows Service or an ISAPI extension, i have deployed many ISAPI extensions built RTC, it is working.

2) It might look intimidating at first but the learning curve is astonishing, just try to understand the demos ( file server, webserver, messenger..) those demos you need to check, once you understood few key things like TRtcInfo, TRtcConnection, Sessions, DataProvider( this has nothing to do with DB there is others components to handle that ) and (Server/Client)Modules with functions, all of that sounds too much but in fact it is event-driven-can't-be-easier-to-use, then you can build very powerful client/server applications in record time.

3) You clients can use an app to upload ( built by you ) or use any Web Browser at hand.

4) In case you want to ditch HTTPS with it, it does have decent and sufficient encryption and security even if the traffic is plain HTTP, but in this case you lose the Web Browser usage.

5) RTC is cross platform out of the box.

 

That few things came to my mind.

 

Now away from RTC or HTTP, i think you should do your diligence (as you asked this question it does meant you are not familiar with FTP, right ?), so keeping an open mind and hear others suggestions in this forum, and ONLY use what you feel comfortable with.

[Remy Lebeau 1255]

7 hours ago, Mark Williams said:

Uploads

The user requests permission for upload via my webserver and if authenticated is issued with a unique token to be stored in the database along with an expiry time which the user will need to renew periodically via the webserver and an identifier to identify which folder(s) the user is allowed to upload to.

If you are using HTTP for authentication, then why not just keep using HTTP for the file transfers, too?

7 hours ago, Mark Williams said:

The FTP client would submit the token to the FTP server which would retrieve the token, expiry  time and folder identity prior to permitting any upload request.

That is doable.  However, there is nothing in the FTP protocol to handle such tokens, so that will have to be transmitted as the username/password/account for the session.  But once authenticated, you can set the TIdFTPServerContext.HomeDir and TIdFTPServerContext.CurrentDir properties as needed.

7 hours ago, Mark Williams said:

When the client has finished it will request the deletion of the token.

There is nothing in the FTP protocol to handle that.  I would suggest simply deleting the token from the database as soon as a client logs in with it, making it a session token.  That way, noone else can login with that same token, and it will be invalidated once the client disconnects.

7 hours ago, Mark Williams said:

There are likely often to be large numbers of files to be uploaded so preferably the FTP Server will be able to store the token, expiry time etc rather than querying the database for every upload.

Yes, you can store custom data in the TIdFTPServerContext on a per-client basis.  You definately do not want to query the database on every transfer.

7 hours ago, Mark Williams said:

The FTP client would submit the token on every upload request.

That will not work.  There is nothing in the FTP protocol that would allow for such data on a per-request basis.  But, the client can login with the token, and the server can validate it at the time of login and remember that the client is allowed to upload, and then the client can freely upload files as needed.

7 hours ago, Mark Williams said:

The FTP Server would check periodically whether the token has been removed.

Not necessary if the token is deleted as soon as it is redeemed during login.  However, if would be trivial to remember the expire time on a per-client basis, and the server could validate the expiration on each file transfer, if needed.

7 hours ago, Mark Williams said:

Downloads

There may be a requirement for download of a large batch of files. So similar approach to the above.

Same restrictions apply as above.

7 hours ago, Mark Williams said:

I already keep a record of every file in a database so each has a unique id and there is another table which records user rights to access a given file. Before submitting download request user applies for token via webserver and at the same time submits a list of ids of those file he intends to download. Webserver checks which ones user has access to and populates a table with the file ids ("permitted_files"). On receiving the retrieve request the FTP does the same as for upload, but additionally queries the "permitted_files" table to see which files the user is permitted to download.

Well, then now you are back to hitting the database on each file transfer, which is not a good thing for performance.

7 hours ago, Mark Williams said:

Assuming it is sensible, what do I need to implement this within the FTPServer component. My best guess at the moment is that it would be handled via CommandHandlers. 

You don't need to use CommandHandlers for this.  TIdFTPServer does that internally for you.  TIdFTPServer has OnUserLogin, OnUserAccount, OnStoreFile, and OnRetrieveFile events that you can use instead.

7 hours ago, Mark Williams said:

1. No GUI interface is required and I don't really need a command line so how best to implement this on the Server: as a Windows service? 

Yes, a windows service would be a good idea.

7 hours ago, Mark Williams said:

As the FTP Server may be receiving numerous requests at the same time how best to manage the data relative to specific requests (ie tokens, permitted file ids etc).

That is a very broad question with many possible different solutions.

[Fr0sT.Brutal 897]

In your scheme with tokens FTP is useless IMHO. Do everything with HTTP. You'd have to use custom clients anyway (to utilize token stuff) so no sense in FTP at all.

 

Alternatively, you can really deal with temporary logins and folders personal for each user. This way FTP is fine and any standard client could be used. Expiration could be done by the moment of last activity

[Mark Williams 13]

Thanks to everyone for the detailed responses and suggestions.

 

I think my fundamental error was my assumption (long-held and it seems with little basis) that FTP is faster than HTTP for multiple file transfers.

 

I have already written an ISAPI dll using Delphi's TWebBroker which seems to be working well for my needs I just thought I could speed things up with FTP. From the above responses a better approach may be to more finely tune my ISAPI dll.

 

The comments re Real Thin Client are noted and I will certainly look into this at a later point.

 

In short, thanks for the feedback which it appears has saved me a potential mountain of unncessary work!