Angus Robertson

Know nothing about IntraWeb, what IP addresses did you configure it to listen on? 

There are several ICS samples that illustrate simple TCP connect.  For simple telnet, you would be better off using TIcsIpStrmLog than TWSocket, much faster and easier to use, see sample OverbyteIcsIpStmLogTst,dpr. 

Angus

sgcWebSockets  does include an OAuth2 server, for 600 euros. 

Angus

OpenSSL 3.0 (and 3.0.1, 3.0.2 etc) has now been designated a Long Term Support release, with security and bug fixes until 7th September 2026.  ICS users are recommended to update to V8.68 or later and OpenSSL 3.0 for long term support. 

The current LTS release 1.1.1 will continue to be supported until 11th September 2023.  The older 1.0.2 release only receives security fixes if you have a paid support contract with OpenSSL. 

The next main release will be OpenSSL 3.1.0 with initial support for QUIC which is needed for HTTP/3, but it may take a couple of years for all the work to be completed, they plan to release updates every six months.

Angus

To clarify, this is down to DNS.  If a DNS lookup returns both IPv4 and IPv6 addresses, the client will usually try IPv6 first.  The Windows HOSTS file has both IPv4 and IPv6 lookups for localhost so servers need to listen on both addresses.  Can be fixed by removing the IPv6 address from HOSTS. 

Angus

Quote

After we made our local server listen for ipv6 connections as well, that timeout was gone.

Yes, I've seen that.  If the client is allowed to use IPv6 and IPv4, it will try IPv6 first, timeout and try IPv4.  All client software should have a setting to make it use IPv4 only, in ICS it's SocketFamily.  

Angus

That diagram suggests the cloud service is running both authentication and API services, really need a version where they are separate. 

It should not be that difficult to design an authentication REST server using ICS or Indy, clearly it would need a database behind it which takes it beyond the type of samples ICS offers,  and into end user application territory.   It would also need a protocol between the authentication and API servers, not sure if there is a standard for that, guess that is Okta's market. 

Angus

Not really look at the server OAuth2 implementations, but isn't it usually a separate REST server than handles authentication, returning a bearer token which is passed to the API server, which presumably then checks back locally with the authentication server to validate it?

Often the authentication server is run by a separate company, like Okta. Started looking at Okta once, but the level of emails and calls I got put me off.  

Angus

The ICS fork is at: https://svn.overbyte.be/svn/ics/trunk/Source/OverbyteIcsSuperObject.pas and https://svn.overbyte.be/svn/ics/trunk/Source/OverbyteIcsSuperXMLParser.pas

There is a sample application OverbyteIcsJoseTst.dpr that illustrates parsing Json and XML into a grid at multiple levels.  The OverbyteIcsHttpRestTst.dpr sample does the same thing for Json APIs over the internet.  The OverbyteIcsSslX509Certs.pas unit makes extensive use of SuperObject to create and parse Json for SSL certificate orders.

Angus

ICS includes forked version of SuperOject 1.2 so that ICS can offer the same JSON support on all platforms, It's not the latest version, since ICS already has time zone functionality that got added to SO.  Also removed TSuperWriterSock since did not need it. 

I've added better TDateTime support, reporting JSON parse errors and other minor fixes.  Fully compatible with D7 to D11.0.

The unit's only dependency is an ICS utility unit, so it can be used without anything else.

Angus

I set different levels when checking the password, that determine what data the execute events are allowed to access.   I will check the sample works the same with GET and POST in a day or two. 

Angus

Yes, ICS should build on Linux, but is awaiting a Linux expert to update the MacOS message handler code to proper Linux. 

Angus

I thought you were writing an HTTP client, not a server, how would code to create chunks help you? 

ICS does not contain any server code or samples to create chunked data, it is normally created by the application. 

The ICS HTTP client and HTTP proxy both unchunk received data into a stream.  The point I was making was the proxy component can accept chunked data and forward it as unchunked, which I thought was your original question.  I'd have thought the Indy client would unchunk as well. 

Angus

My main TUrlHandler.Execute functions start with:

if (Client.AuthUserName <> '') then begin

to check for authentication, but the server does the main authentication in the onCheckPassword event which returns false for failure, but does not clear AuthUserName.  So there may be a bug for POST.  Sorry, don't have time to test it this week.

Angus

If Authentication fails, the client Flags property is set to hg401 so that response is finally returned.  The code and sample web servers have checks for h401 in many places, I'd guess your code is checking the flag for GET for not for POST, somewhere.  The component does not stop calling events because authentication has failed.  Any of those events may also set Flags to hg401 or something else. 

Angus 

Chunking an HTTP response is a server decision, there is no way a client can stop it.  Some web applications are simply written to return the response as lots of small chunks instead of building the response locally and sending it once the length is known. 

One web site I access daily recently changed so that all file downloads now come as a single chunk, this is annoying since there is no HTTP content size header and you've no idea how much is arriving. 

The only real solution is to access the site through an HTTP proxy that unchunks the response and fowards it as sized content, the ICS proxy does this. 

Angus

Your first suggestion was correct, that file only contains the data just written to it, a few bytes.

Angus

Thanks, both fixes done in my local copy, will be in SVN in a few days.

Angus

Sorry if you think my replies are insulting.  But you are asking the same question more than once, and ignored my previous reply that answered that question, and previous replies that suggested simpler, quicker ways of doing what we think you are trying to do.  If you don't want our advice, please don't keep asking for it.  

Angus

You are told previously that OnDataAvailable is called repeatedly, each time returning up to a few thousand bytes of data, you only know all the data has arrived when the connection is closed or your protocol says so.  

You have been advised to use proper high level components that do this work for you, but you seem to want to reinvent these components, without listening to our advice. 

Angus

So you design your protocol to identify where each connection comes from, or work it out from the content of the data.  ICS has no idea.

Angus

How you identify a connection depends on the protocol you are using, usually they will identify themselves in some way if they expect a customised response. 

Without a protocol, all you have is the remote IP address and port.

Your various posts here all suggest you should be using the HTTP components, which compress data and send headers and have authentication.

Unless you are expecting to handle hundreds of new connections each second, TWSocketThrdServer is not a good component to use, it is ancient and has not been tested for 10 years, there is no sample application using it.  TWSocketServer is the component used by all ICS server applications.

Angus

Exactly the same as sending an uncompressed memory stream, TWsocket has no idea what type of data you are sending.

If you mean how do you compress data in Delphi, there are lots of ways.  The ICS web server uses ZLIB to compress HTTP responses.

Angus

Thanks, not sure why the content checking is so strict, but added your fix to my copy, will be in SVN with the next batch of updates next week.

Angus

ICS SMTP access also supports OAuth2 authentication as standard, which may be required for some Microsoft accounts.

Angus

You should be using the TIcsMailQueue component, there is a working mass mailer example in OverbyteIcsMailQuTst.dpr. 

You prepare mail using TSslSmtpCli, the queue component saves it as an EML file and places it into a queue from which a thread tries to send it repeatedly to multiple SMTP servers, and when sent successfully the EML file is either deleted or archived.   At least I hope it's the correct EML format, I've never tested with other applications. 

Angus