Angus Robertson

I thought you were writing an HTTP client, not a server, how would code to create chunks help you? ICS does not contain any server code or samples to create chunked data, it is normally created by the application. The ICS HTTP client and HTTP proxy both unchunk received data into a stream. The point I was making was the proxy component can accept chunked data and forward it as unchunked, which I thought was your original question. I'd have thought the Indy client would unchunk as well. Angus

My main TUrlHandler.Execute functions start with: if (Client.AuthUserName <> '') then begin to check for authentication, but the server does the main authentication in the onCheckPassword event which returns false for failure, but does not clear AuthUserName. So there may be a bug for POST. Sorry, don't have time to test it this week. Angus

If Authentication fails, the client Flags property is set to hg401 so that response is finally returned. The code and sample web servers have checks for h401 in many places, I'd guess your code is checking the flag for GET for not for POST, somewhere. The component does not stop calling events because authentication has failed. Any of those events may also set Flags to hg401 or something else. Angus

Chunking an HTTP response is a server decision, there is no way a client can stop it. Some web applications are simply written to return the response as lots of small chunks instead of building the response locally and sending it once the length is known. One web site I access daily recently changed so that all file downloads now come as a single chunk, this is annoying since there is no HTTP content size header and you've no idea how much is arriving. The only real solution is to access the site through an HTTP proxy that unchunks the response and fowards it as sized content, the ICS proxy does this. Angus

Your first suggestion was correct, that file only contains the data just written to it, a few bytes. Angus

Thanks, both fixes done in my local copy, will be in SVN in a few days. Angus

Sorry if you think my replies are insulting. But you are asking the same question more than once, and ignored my previous reply that answered that question, and previous replies that suggested simpler, quicker ways of doing what we think you are trying to do. If you don't want our advice, please don't keep asking for it. Angus

You are told previously that OnDataAvailable is called repeatedly, each time returning up to a few thousand bytes of data, you only know all the data has arrived when the connection is closed or your protocol says so. You have been advised to use proper high level components that do this work for you, but you seem to want to reinvent these components, without listening to our advice. Angus

So you design your protocol to identify where each connection comes from, or work it out from the content of the data. ICS has no idea. Angus

How you identify a connection depends on the protocol you are using, usually they will identify themselves in some way if they expect a customised response. Without a protocol, all you have is the remote IP address and port. Your various posts here all suggest you should be using the HTTP components, which compress data and send headers and have authentication. Unless you are expecting to handle hundreds of new connections each second, TWSocketThrdServer is not a good component to use, it is ancient and has not been tested for 10 years, there is no sample application using it. TWSocketServer is the component used by all ICS server applications. Angus

Exactly the same as sending an uncompressed memory stream, TWsocket has no idea what type of data you are sending. If you mean how do you compress data in Delphi, there are lots of ways. The ICS web server uses ZLIB to compress HTTP responses. Angus

Thanks, not sure why the content checking is so strict, but added your fix to my copy, will be in SVN with the next batch of updates next week. Angus

ICS SMTP access also supports OAuth2 authentication as standard, which may be required for some Microsoft accounts. Angus

You should be using the TIcsMailQueue component, there is a working mass mailer example in OverbyteIcsMailQuTst.dpr. You prepare mail using TSslSmtpCli, the queue component saves it as an EML file and places it into a queue from which a thread tries to send it repeatedly to multiple SMTP servers, and when sent successfully the EML file is either deleted or archived. At least I hope it's the correct EML format, I've never tested with other applications. Angus

SFTP and FTPS can mean anything people want to mean, using those letters without explanation is confusing. Angus

If by SFTP you mean secure FTP rather than Simple FTP (a UDP protocol), ICS has a sample OverbyteIcsSslMultiFtpServ.dpr which is really designed to be a Windows service application. It supports multiple SSL hosts with multiple listeners, can order it's own SSL certificates and will create self signed certificates for any missing, and will email status information and errors to an administrator. This sample is really a commercial FTP server and just needs the addition of user accounts. Angus

Your TMyRec would not work as a header since the String types point to memory and only the pointer would be sent, it needs fixed length elements and ideally Length: Integer at the start set to the actual length, so the application decoding it can handle changes in the record in the future. Angus

As we explained previously, you need to design a protocol with a header to explain what type of data you are sending and how much. One well known such protocol is HTTP which ICS supports, and is much easier to use than designing your own. One standard in most protocols is that text is terminated with CRLF at the end of each command or line, which your SendStr does not do, but which can be useful to know that a complete line has arrived for processing and potentially to recognize that text is being sent, although binary files may potentially include CRLF. Angus

Magenta Hardware Components is a set of six main components for Delphi 2007 to Delphi 11.0 and later, as follows: 1 - Magenta Serial Port Detection Component which contains serial COM port enumeration functions, using several methods which can identify different ports depending on how they are installed, all are combined and a sorted array returned with friendly names and install information. 2 - Magenta Hardware Events Component that listens for Windows hardware event messages and calls events handlers for device changes such as serial ports, disk volume changes, low disk space events and power events. 3 - Directory Changes Monitoring Component, that notifies changes in a directory such as file or directory Create/Delete/Modify/Rename. 4 - Magenta GPS and Location Component is designed to process GPS location data from various sources with an event triggered when movement is detected. GPS sources supported include Windows Location API, NMEA 0183 sentences, GT02 GPS Tracker Protocol, TK102/103 Tracker Protocol and WondeX/TK5000 Tracker Protocol. Sample shows movement track on a Google map. 5 - Magenta Firewall Component has functions to search and list selected Windows Defender Firewall rules and settings, and to add and remove such rules. There is also some code that may be used in Inno Setup scripts to set-up firewall rules during application installation. 6 - Magenta Check Disk and Format Disk component used to format fixed or removable disk drives and perform disk checks on Windows NT4 and later. More details and download from: https://www.magsys.co.uk/delphi/maghardware.asp Or SVN: https://svn.magsys.co.uk/svn/maghardware/ Magenta Hardware Components are freeware, but are still copyrighted by Magenta Systems Ltd who may change the status or withdraw it at any time, without notice. Angus

ICS V8.68 has been released at: http://wiki.overbyte.eu/wiki/index.php/ICS_Download ICS is a free internet component library for Delphi 7, 2006 to 2010, XE to XE8, 10 Seattle, 10.1 Berlin, 10.2 Tokyo, 10.3 Rio, 10.4 Sydney and 11.0 and C++ Builder 2006 to XE3, 10.2 Tokyo, 10.3 Rio, 10.4 Sydney and 11.0. ICS supports VCL and FMX, Win32, Win64 and MacOS 32-bit targets. The distribution zip includes the latest OpenSSL 3.0.1 win32, with other versions of OpenSSL being available from the download page. Major Changes in ICS V8.68 include: 1 - V8.68 is a minor release, mainly HTTP client and server improvements including new request and response headers to assist browser caching and conditional requests, improved error reporting to help diagnose failed HTTP requests, and improvements in the HTTP REST component allowing it to save files including resuming failed downloads and download files of any size. There are minor fixes in various components, and updates to installing on MacOS and C++ Builder. There is also a new File Clean-Up demo ideal for deleting the old log files that many applications (including some ICS samples) leave on systems. 2 -The HTTP client TSslHttpCli now keeps the Etag response header as ResponseEtag, allowing applications to save it with the page content, and when requesting a refresh to add the request method ReqIfNoneMatch or ReqIfMatch to avoid downloading the page again. Improved RequestDoneError to give more information than a simple abort, added httperrOutOfMemory and httperrBgException which happen while receiving and processing data in the OnDocData event and httperrSslHandShake. For range downloads, added the ContentIfRange request header which can send an RFC1123 date or Etag so a partial download only happens if the file is unchanged. 3 - Previously, the HTTPS REST client TSslHttpRest always downloaded content to a TMemoryStream with content size being limited to MaxBodySize (default 100 MByte), and generally restricted by memory to less than 250 MByte. To remove this limitation, added the HttpMemStrategy property with THttpMemStrategy on how to handle downloads: HttpStratMem only TMemoryStream; HttpStratTemp uses a work file in the system temporary directory for sizes larger than MaxBodySize; HttpStratFile always writes a named file HttpDownFileName (with .part extension during download); HttpStratResume is similar to HttpStratFile but supports resume of failed partial downloads (with .http extension for resume information). Property ResumeMinSize defines the minimum sized partial file that should be resumed, rather than start again (default 64K). Added ShowProgress property that causes download information to be sent to the OnHttpRestProg event. 4 -The HTTP server TSslHttpServer now supports the If-Range, If-Match and If-None-Match request headers for conditional pages using Etag or last modified date, and sends 304 not modified for matches, to help with caching. Added an ETag header to responses in AnswerStream and AnswerPage when we can create one from a file modification date and size (base64 CRC32), or if the EntityTag property is specified in the client onGetDocument event before using hgSendDoc or hgSendStream, perhaps a CRC32 of the entire content from a cache. The 304 not modified response now includes more recommended headers. Also add a Date: header to AnswerStream and AnswerPage responses to help with caching. 5 - The Multi HTTPS client TIcsHttpMulti has been updated to a fix a problem that meant large files failed download with only an abort error if too large for TMemoryStream, now downloaded to TFileStream with .part extension and renamed up successful completion. 6 - The previous ICS release added support for the OpenSSL 3.0 release. Now it's been available for three months it has been added to the main distribution, the samples SslInternet directory now has both OpenSSL 1.1.1m and 3.0.1. ICS now supports YuOpenSSL 3.0 and 1.1.1 versions as commercial DCUs allowing applications to be used with OpenSSL without needing separate DLLs. 7 - In TIcsFileCopy, the DeleteFiles method now supports a new Zipped property so files are zipped with .zip extension before being deleted, useful for cleaning up old logs to save space (requires VCLZip). The OverbyteIcsXferTst,dpr sample has a new File Clean-Up tab to test the multiple DeleteFiles method, it allow files older than x days or a specific date range to be archived/zipped (to save space) or deleted, ideal for deleting the old log files that many applications (including some ICS samples) leave on systems. 8 - In the FTP client TSslFtpClient, using the PORT command to set Active mode now prevents other FTP clients sharing the same port number and address, usually only a problem with a small port pool (which is not recommended). 9 - Updated the trusted root certificate bundle files, lots of changes from Microsoft since June, Google is now issuing it's own certificates. Updated the build-in sslRootCACertsBundle, few gone, now total 59 certificates commonly used. 10 - ICS should now build with C++ for RAD Studio 10.4 and 11.0, fixed the 11.0 packages and various Windows API related units, including for Win64. More detailed release notes are at http://wiki.overbyte.eu/wiki/index.php/ICS_V8.68

Thanks for the fixes, I've done 3, 4 and 5, all my mistakes adding new features for Windows. Will be in SVN shortly. But I don't understand 1 and 2, can not see why there would be a fatal error on line 162 or what fix you actually made to stop it. Angus

@anonymousaccount I've fixed the FTP server bug, you found the correct location of the bug but the fix was slightly different. Should not matter now since I assume you are using the new FTP server sample with IcsHosts. Will be in SVN shortly. Angus

I know, really can not be bothered with modern styles, it's content that matters to me! Sorry, should really have posted this in third party, did not realise there was a main forum as well as the sub-forums. Angus

We don't test ICS on MacOS, since the developers don't have access to any Apple hardware. So we are dependent upon contributors here to test ICS on MacOS and fix any bugs, which we then incorporate in the distribution. You may find using a much older version works better, except for obsolete OpenSSL versions. And we don't provide OpenSSL for MacOS either. I'll fix the TMsCertTools error, that is Windows only so should not build on MacOS. I'll look at the others, but they are not Windows specific. Angus

The onDataAvailable event is called repeated as more data arrives, you need to keep adding data to the receive stream until the connection is closed or your protocol signals the stream has received all that intended (why there are headers in HTTP). TWSocketThrdClient and TWSocketClient relate to ICS server components and won't help in this case. Angus