HTMLValidator.com

4 hours ago, Angus Robertson said:

The IcsShellExec function moved units in V8.69, it is now in OAuth, but perhaps you have an old HttpRest unit with it still in?  Or an old DCU.

 

When installing a new version of ICS, it's always safest to use the Build Group pane to build all four versions of all packages, just wish it installed them as well. 

 

Angus

Thank you. Must have been some old junk laying around somewhere. Cleaned out old files in C:\Users\Public\Documents\Embarcadero\Studio\22.0 and eventually got everything to install... so far so good! Although my 64-bit app builds are crashing (32-bit works)... but I don't think it's an ICS issue.... time to figure that out now... it's always something, especially with C++Builder. 🙂

Was able to download 8.70 (Delphi 11 only)... but I'm trying to install in Delphi 11.2 (with all C++ Builder files generated because that's what I use) and I'm getting errors: [dcc32 Error] OverbyteIcsSslHttpOAuth.pas(897): E2251 Ambiguous overloaded call to 'IcsShellExec'. Searched for IcsShellExec in the forum and came up with nothing.

Awesome, thanks.

But the download the link is not working... any ideas why? A temporary issue I hope.

6 hours ago, Daniele Teti said:

Any news on this? Did you got it to work?

I "played" with it briefly but had to move on to other things. Never got it to work. Maybe I'll try again and spend more time on it when I eventually upgrade to Rad Studio 11... or by some "magic" maybe they will fix C++ Code Insight in 11 and I won't have to waste time on it..

Great, thank you for the fix.

I am not sure I understand.

I have never used any certificate checking.

If I request https://www.w3.org/TR/wai-aria-1.1/#aria-atomic directly, then it works fine. If there is a redirect to that URL, then I get a 400 error.... I'm thinking because it's not removing the fragment part (#aria-atomic).

This is what my log looks like:

Proxy> No proxy being used
Request> GET /help.php?h=aria-atomic&m=6 HTTP/1.0
Request> Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Request> Connection: Keep-Alive
Request> User-Agent: CSS HTML Validator Professional (https://www.htmlvalidator.com/)
Request> Host: www.htmlvalidator.com
Request> 
SslHandshakeErrCode> 0 (0 if no error)
SslHandshakeRespMsg> SSL Connected OK with TLSv1.3, cipher TLS_AES_256_GCM_SHA384, encryption AESGCM(256), message auth AEAD
Response> HTTP/1.1 302 Found
Response> Date: Sat, 10 Jul 2021 18:53:46 GMT
Response> Server: Apache
Response> Referrer-Policy: no-referrer-when-downgrade
Response> X-Content-Type-Options: nosniff
Response> X-Frame-Options: SAMEORIGIN
Response> X-Xss-Protection: 1; mode=block
Response> Content-Security-Policy: 
Response> Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Response> Upgrade: h2
Response> Connection: Upgrade, Keep-Alive
Response> Location: https://www.w3.org/TR/wai-aria-1.1/#aria-atomic
Response> Vary: User-Agent
Response> Content-Length: 0
Response> Keep-Alive: timeout=5, max=100
Response> Content-Type: text/html; charset=UTF-8
Location> Changing to: https://www.w3.org/TR/wai-aria-1.1/#aria-atomic
Request> GET /TR/wai-aria-1.1/#aria-atomic HTTP/1.0
Request> Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Request> Connection: Keep-Alive
Request> User-Agent: CSS HTML Validator Professional (https://www.htmlvalidator.com/)
Request> Host: www.w3.org
Request> 
SslHandshakeErrCode> 0 (0 if no error)
SslHandshakeRespMsg> SSL Connected OK with TLSv1.3, cipher TLS_AES_128_GCM_SHA256, encryption AESGCM(128), message auth AEAD
Response> HTTP/1.1 400 Bad Request
Response> date: Sat, 10 Jul 2021 18:53:47 GMT
Response> last-modified: Thu, 04 Jun 2020 15:34:04 GMT
Response> etag: "420-5a743dfdcf300"
Response> accept-ranges: bytes
Response> content-length: 1056
Response> content-type: text/html; charset=iso-8859-1
Response> x-backend: ssl-mirrors
Response> strict-transport-security: max-age=15552000; includeSubdomains; preload
Response> content-security-policy: upgrade-insecure-requests
Response> connection: keep-alive

-- Request Done --
StatusCode   = 400
ReasonPhrase = Bad Request
Time         = 516 ms

Sorry about that.

Please try https://www.htmlvalidator.com/help.php?h=aria-atomic&m=6 instead of https://www.htmlvalidator.com/help.php?h=aria-atomic&m=6

The "&" should be "&". Not sure if that was my mistake or something trying to be smart and converting the "&" to "&" when I copy and pasted the link.

Thanks.

Trying to download a URL with TSslHttpCli.
 

URL is: https://www.htmlvalidator.com/help.php?h=aria-atomic&m=6

Which redirects to: https://www.w3.org/TR/wai-aria-1.1/#aria-atomic

Which results in a "HTTP/1.1 400 Bad Request".

I think the 400 is because TSslHttpCli is not removing the fragment (#aria-atomic) when it is requesting the redirected URL.

Real browsers work fine.

Is this a bug?

On 4/10/2021 at 3:34 AM, Angus Robertson said:

I did wonder whether to write a check URL function, but it's not easy, except for space which is illegal in all URLs.  Lots of other special characters like /, & and % may be the result of previous encoding, so you don't know whether to encode them again.

 

But open to suggestions for a URL checker.

Well, you wouldn't be able to detect some issues but a URL/percent encoding checker that would check for invalid encoding like when there is % that is not followed by two hexadecimal digits, and of course if there is that invalid space character. Possibly also detect encoded control characters like %00-%1F. I may have already written something like this a while back (I have to check as I can't remember for sure) but I thought that a function like this might already be in ICS.

Not a big issue but perhaps something to consider.

13 hours ago, Lajos Juhász said:

The available information can be found at: http://docwiki.embarcadero.com/RADStudio/Sydney/en/Code_Insight_Reference

You can always try to enable logging and read the logfiles.

Thanks.... I'll start there and see if I can finally get it to work.

If anyone else has any other ideas or resources then please share.

Also, I'm still interested if it works for anyone here for any "real world" project.

Doesn't even work for me... it just says "CodeInsight: Stopped".

Does Code Insight even work for any C++ users with "real-world" projects?

Every time I update I hold out a little hope that Code Insight will start working again for my real-world C++ projects.

Unfortunately, this hasn't happened in years.

When I hit Ctrl+Space, it very briefly changes the mouse pointer but does nothing. In the "Projects" Tool Window, there's a "Code Insight Activity" bar. What the heck is that suppose to do? Mine seems to always show "CodeInsight: Stopped".

Is there a link that explains everything to check for in order to get Code Insight to work in C++? For example, all the requirements for it to work... and things to avoid that might break it. And it would be extremely useful if there was some way to see any messages that "Code Insight" might generating as to why it is failing.

Also, if anyone has a real-world C++ project and Code Insight actually works for you, then please let me know. I'm curious if it works for anything other than tiny demo apps.

7 hours ago, Angus Robertson said:

Yes, paths are meant to be encoded within the path delimiters, not / itself, unless after ?  So I did the simple fix of only handling space.  About 20 links failed my own tester with full encoding.  So a new version is in SVN.  That Canadian site also broke the ICS proxy due to not supporting absolute URLs used by proxies, that's been on my list to fix for a year, so got done as well. Testing is always useful, gets me to fix things. 

Thanks. I've updated and it fixes the issue I mentioned.

As you suggested, I'm also using onHeaderData to get the "raw" location URL (before encoding fixes/changes)... and basically throwing a warning if a location change URL contains a space character.

While I have your attention... is there a quick "ICS" way/function to check a URL to see if there are any encoding issues with it? Right now I'm only checking for space characters (very basic).

8 hours ago, Angus Robertson said:

Further to my last comment, I need to change the auto URL encoding for relocation so it does not process a URL that is already encoded correctly.  There will be another version soon.

 

Angus

 

Well, I'm not sure how you are handling this... but when there was a location change to "https://www.htmlvalidator.com/test/contains space.txt" it looks like it encoded the URL to "https://www.htmlvalidator.com/test%2Fcontains%20space.txt", so it fixed the space character but broke the '/' character.

Perhaps you should just change space characters to %20?

11 hours ago, Angus Robertson said:

I found the problem with HEAD and some redirections, it failed to start the redirected request until close was called, a bug that seems to have been there for many years.  Looked at my own link checker and I use GET not HEAD which is why I've never seen it.  Your 404 error page returns content even for 200 and HEAD.  It will be in SVN tonight.

 

Hello,

I downloaded the latest SVN and it looks like the major issue with the link checker stalling is fixed so thank you very much!

A minor issue... it looks like you're fixing the space character in location redirections which I think (in general) is a good idea but because I want my link checker to report issues and errors to the user, is there a way to detect that this fix/correction was done so I can report it to the user?

5 minutes ago, Angus Robertson said:

I found the problem with HEAD and some redirections, it failed to start the redirected request until close was called, a bug that seems to have been there for many years.  Looked at my own link checker and I use GET not HEAD which is why I've never seen it.  Your 404 error page returns content even for 200 and HEAD.  It will be in SVN tonight.

That's great... thanks!

Thanks for checking into this. My link checker should check for spaces in redirected URLs and report them and maybe fix them... I'll work on this, but this is not the main problem.

I am still confused why the RequestDone function is called twice? Shouldn't it be called only once?

The main problem is my link checker stalls out for some reason when this happens, and it never works again until the user exists and restarts my application. Did you find any indication of this issue also causing any serious problems or corruption with the operation of TSslHttpCli? I need to research more what is going on that this link stops/stalls my link checker.

Having the TSslHttpCli correct bad URLs like browsers do seems like a good idea in the cases where it is trivial... but if one is running a link checker like I am, I would want a way to disable these corrections or somehow know that they have been done so I can report the problem to the user.

On 4/7/2021 at 2:30 AM, Angus Robertson said:

Sorry, your redirection problem needs actual testing and not had time in the last two weeks for anything new. Angus

Thank you. Totally understand the time issue. However, this is a big issue for me so I hope that you will be able to find the time to help with it soon. UPDATE: Looks like you responded already... so nevermind... thanks!

Had this same problem when updating to 10.4.2.

Found this old file: C:\Windows\SysWOW64\RaizeComponentsVcl270.bpl

Deleted it. Tried installing again. Got a similar message but for different BPL.

Found this old file: C:\Windows\SysWOW64\RaizeComponentsVclDb270.bpl

Deleted it. Tried installing again. Now getting a "Cannot perform operation" dialog. Forcing shutdown. Reloading. Removing Konopka controls from "Install Packages".

Tried installing again. Success! Finally!

Now I bet it screwed up my 10.3 installation.

Thank you! I just updated to 8.65 and now 8.66 and 1.1.1k as I try to figure out this problem:

Unfortunately the problem persists in 8.66 with 1.1.1k and I'd still love to get that issue resolved.

I can say that 8.66 compiled fine (via Delphi environment) for my VCL usage with C++Builder in Sydney 10.4.1. Of course I had to set all the projects in D104InstallVclFmx.groupproj to generate all C++ files... I did not build the FMX projects though.

I use TSslHttpCli for link checking in my application. I just installed the latest recommended release, 8.65.

A user reported that the link checking in my application is stalling (getting stuck).

This URL seems to be causing a problem: 

http://www.ec.gc.ca/dd-sd/Default.asp?lang=En&n=C2844D2D-1

When I try to check it using TSslHttpCli and HttpCli1->HeadASync(), I get two calls to OnRequestDone.

1. First, there is a location change to "http://ec.gc.ca/Error 404.html" (could the space character in the new URL cause a problem?)

2. Next there is a first RequestDone call with StatusCode 0 and ReasonPhrase "", which I've programmed to ignore because I expect a 2nd call to RequestDone

3. Next there is a second RequestDone call with StatusCode 200 and ReasonPhrase "OK".

4. Something goes wrong here and my link checker stalls/stops and I'm not sure why but I'm wondering if something is messed up / corrupted with TSslHttpCli ... especially since the RequestDone function is being called twice (is this a bug)? Or perhaps I'm doing something wrong?

I'd really appreciate any insight into what might be happening. Thanks!

Thank you all. This is regarding a link checker in my software... so if the Error is 10053 and StatusCode is 200 then I'll just assume the link exists and ignore the 10053 error.

3 hours ago, FPiette said:

WSAECONNABORTED 10053

Software caused connection abort. An established connection was aborted by the software in your host computer, possibly due to a data transmission time-out or protocol error.

Thanks... but if the connection was aborted, why a StatusCode of 200 and ReasonPhrase of "OK"? Perhaps the connection was aborted after the status was received?

Interesting:

https://www.chilkatsoft.com/p/p_299.asp

In the RequestDone event of a TSslHttpCli object, a user is reporting that he's getting Error 10053 but StatusCode is 200 and ReasonPhrase is "OK".

This seems to only happen on his "test" server and not his "production" server. The "test" server has a lot less resources and it's being hit hard with a lot of requests but I'm not sure that explains what's happening.

Why would Error be 10053 but StatusCode and ReasonPhrase indicate no error?

UPDATE: Customer adds: "The test and production servers have same configuration files (Apache + nginx), same settings for SSL/TLS (ciphers...). The difference is that the test server uses self-generated certificate, while the production server has certificate issued by Certificate Authority. Most (if not all browsers) do not like self-generated certificate (or bad or obsolete certificates) and issue warning with such URLs. The problem is that only some links (5000+) got 10053 error, not all (300K+ if I remember well) if I refer to the statistics shown during run, so probably the server load would be part of the problem, not the protocol itself."

Thank you!

5 hours ago, ivp said:

Here is the patch for 1435 revision:

 

ics-CB2009-fix-1435-ALL.patch

Thanks! Hopefully these problems won't be an issue the next time I update ICS.