HTMLValidator.com

10 hours ago, w0wbagger said:

I was not having this issue in previous attempts. I was able at one point to compile and install all 32-bit packages, although could never get my program working.
@HTMLvalidator, were you able to get the groupproj to compile without changing many options? Can you try whatever you did with the overnights, per Angus' request? It would be nice to have a stable, easily installable C++ version upon which future builds could be made that includes all of Angus' significant improvements.
thanks.

I wish I had time to mess with this but I am (again) fighting my development tools to get stable working builds of my app. I've fought many times before and sometimes they are problems with the paths I'm using (and maybe old files laying around) and sometimes bugs with their compiler and/or linker... or maybe using a compiler/linker setting that is buggy or broken or otherwise problematic.

The "good" news is that after this I am probably going to stop spending money on new versions of everything and stick with what I have since sales and hassle don't justify continuing to spend money on upgrading (often buggy) development tools... but I hope to end with being able to go forward with the new C++ toolchain (though I'm thinking there might be some bugs in it causing the current problems I'm having).

Anyway, I always use the Delphi projects (not C++ projects because it seems those are often neglected) to compile components and set the Delphi projects to generate all C++ files and output the files to each platform's own folder - i.e. one folder for Win32, one for Win64, and now one for Win64x. This is what I did to get v9.3 compiled (along with adding the #include's to a header file mentioned above) and it seems to work although I am having other issues but I don't think those issues are caused by ICS.

12 hours ago, w0wbagger said:

Thank you! This is the only thing keeping me from moving to 12.2 (I couldn't get 9.x to compile on it), so I'll try this soon! Did you have to use the 'classic' borland compiler for the 32-bit?

No, I don't use the 'classic' 32-bit compiler anymore.

Oh, when building the ICS projects I also used "..\Lib\$(Config)\$(Platform)$(CC_SUFFIX)\$(ProductVersion)" for the 3 output folders in "Building > Delphi Compiler > Output - C/C++" for "All configurations - All platforms" because they were blank and it was throwing everything into the same folder. Then I had to make sure all the search paths were correct so the compiler and linker could find everything it needed.

I'm still having issues getting everything to worth though but not sure if it's ICS or something else.

Well, after playing around, I added these two lines to OverbyteIcsTypes.hpp (after '#include <System.SysUtils.hpp>') and it worked!

#include <mswsock.h>
#include <ws2ipdef.h>

So hopefully this is helpful. I am still trying to get everything to work but at least it compiled and linked.

I'm trying to move to the new C++ Win64x toolchain in RAD Studio 12.2 Patch 2 and I upgraded to ICS 9.3 but now when I try to build a basic C++ VCL app with just a form and a TSslHttpCli component I get these compiler errors:

With bcc32c:

[bcc32c Error] OverbyteIcsTypes.hpp(74): unknown type name '_TRANSMIT_FILE_BUFFERS'
[bcc32c Error] OverbyteIcsTypes.hpp(76): unknown type name '_TRANSMIT_FILE_BUFFERS'
[bcc32c Error] OverbyteIcsTypes.hpp(78): unknown type name 'ip_mreq'
[bcc32c Error] OverbyteIcsTypes.hpp(80): unknown type name 'ip_mreq'
[bcc32c Error] OverbyteIcsTypes.hpp(94): unknown type name 'sockaddr_in6'
[bcc32c Error] OverbyteIcsTypes.hpp(96): unknown type name 'sockaddr_in6'
[bcc32c Error] OverbyteIcsTypes.hpp(98): unknown type name 'ipv6_mreq'
[bcc32c Error] OverbyteIcsTypes.hpp(100): unknown type name 'PIPV6_MREQ'
[bcc32c Error] OverbyteIcsUtils.hpp(624): unknown type name 'sockaddr_in6'
[bcc32c Error] OverbyteIcsUtils.hpp(624): expected unqualified-id

With bcc64x:

[bcc64x Error] OverbyteIcsTypes.hpp(74): unknown type name '_TRANSMIT_FILE_BUFFERS'
[bcc64x Error] OverbyteIcsTypes.hpp(76): unknown type name '_TRANSMIT_FILE_BUFFERS'
[bcc64x Error] OverbyteIcsTypes.hpp(78): unknown type name 'ip_mreq'
[bcc64x Error] OverbyteIcsTypes.hpp(80): unknown type name 'ip_mreq'
[bcc64x Error] OverbyteIcsTypes.hpp(94): unknown type name 'sockaddr_in6'
[bcc64x Error] OverbyteIcsTypes.hpp(96): unknown type name 'sockaddr_in6'
[bcc64x Error] OverbyteIcsTypes.hpp(98): unknown type name 'ipv6_mreq'
[bcc64x Error] OverbyteIcsTypes.hpp(100): unknown type name 'PIPV6_MREQ'
[bcc64x Error] OverbyteIcsUtils.hpp(614): unknown type name 'sockaddr_in6'

Any solution or should I try to go back to a previous version like 8.70 which was working before?

4 hours ago, Angus Robertson said:

The IcsShellExec function moved units in V8.69, it is now in OAuth, but perhaps you have an old HttpRest unit with it still in?  Or an old DCU.

 

When installing a new version of ICS, it's always safest to use the Build Group pane to build all four versions of all packages, just wish it installed them as well. 

 

Angus

Thank you. Must have been some old junk laying around somewhere. Cleaned out old files in C:\Users\Public\Documents\Embarcadero\Studio\22.0 and eventually got everything to install... so far so good! Although my 64-bit app builds are crashing (32-bit works)... but I don't think it's an ICS issue.... time to figure that out now... it's always something, especially with C++Builder. 🙂

Was able to download 8.70 (Delphi 11 only)... but I'm trying to install in Delphi 11.2 (with all C++ Builder files generated because that's what I use) and I'm getting errors: [dcc32 Error] OverbyteIcsSslHttpOAuth.pas(897): E2251 Ambiguous overloaded call to 'IcsShellExec'. Searched for IcsShellExec in the forum and came up with nothing.

Awesome, thanks.

But the download the link is not working... any ideas why? A temporary issue I hope.

6 hours ago, Daniele Teti said:

Any news on this? Did you got it to work?

I "played" with it briefly but had to move on to other things. Never got it to work. Maybe I'll try again and spend more time on it when I eventually upgrade to Rad Studio 11... or by some "magic" maybe they will fix C++ Code Insight in 11 and I won't have to waste time on it..

Great, thank you for the fix.

I am not sure I understand.

I have never used any certificate checking.

If I request https://www.w3.org/TR/wai-aria-1.1/#aria-atomic directly, then it works fine. If there is a redirect to that URL, then I get a 400 error.... I'm thinking because it's not removing the fragment part (#aria-atomic).

This is what my log looks like:

Proxy> No proxy being used
Request> GET /help.php?h=aria-atomic&m=6 HTTP/1.0
Request> Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Request> Connection: Keep-Alive
Request> User-Agent: CSS HTML Validator Professional (https://www.htmlvalidator.com/)
Request> Host: www.htmlvalidator.com
Request> 
SslHandshakeErrCode> 0 (0 if no error)
SslHandshakeRespMsg> SSL Connected OK with TLSv1.3, cipher TLS_AES_256_GCM_SHA384, encryption AESGCM(256), message auth AEAD
Response> HTTP/1.1 302 Found
Response> Date: Sat, 10 Jul 2021 18:53:46 GMT
Response> Server: Apache
Response> Referrer-Policy: no-referrer-when-downgrade
Response> X-Content-Type-Options: nosniff
Response> X-Frame-Options: SAMEORIGIN
Response> X-Xss-Protection: 1; mode=block
Response> Content-Security-Policy: 
Response> Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Response> Upgrade: h2
Response> Connection: Upgrade, Keep-Alive
Response> Location: https://www.w3.org/TR/wai-aria-1.1/#aria-atomic
Response> Vary: User-Agent
Response> Content-Length: 0
Response> Keep-Alive: timeout=5, max=100
Response> Content-Type: text/html; charset=UTF-8
Location> Changing to: https://www.w3.org/TR/wai-aria-1.1/#aria-atomic
Request> GET /TR/wai-aria-1.1/#aria-atomic HTTP/1.0
Request> Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Request> Connection: Keep-Alive
Request> User-Agent: CSS HTML Validator Professional (https://www.htmlvalidator.com/)
Request> Host: www.w3.org
Request> 
SslHandshakeErrCode> 0 (0 if no error)
SslHandshakeRespMsg> SSL Connected OK with TLSv1.3, cipher TLS_AES_128_GCM_SHA256, encryption AESGCM(128), message auth AEAD
Response> HTTP/1.1 400 Bad Request
Response> date: Sat, 10 Jul 2021 18:53:47 GMT
Response> last-modified: Thu, 04 Jun 2020 15:34:04 GMT
Response> etag: "420-5a743dfdcf300"
Response> accept-ranges: bytes
Response> content-length: 1056
Response> content-type: text/html; charset=iso-8859-1
Response> x-backend: ssl-mirrors
Response> strict-transport-security: max-age=15552000; includeSubdomains; preload
Response> content-security-policy: upgrade-insecure-requests
Response> connection: keep-alive

-- Request Done --
StatusCode   = 400
ReasonPhrase = Bad Request
Time         = 516 ms

Sorry about that.

Please try https://www.htmlvalidator.com/help.php?h=aria-atomic&m=6 instead of https://www.htmlvalidator.com/help.php?h=aria-atomic&amp;m=6

The "&amp;" should be "&". Not sure if that was my mistake or something trying to be smart and converting the "&" to "&amp;" when I copy and pasted the link.

Thanks.

Trying to download a URL with TSslHttpCli.
 

URL is: https://www.htmlvalidator.com/help.php?h=aria-atomic&m=6

Which redirects to: https://www.w3.org/TR/wai-aria-1.1/#aria-atomic

Which results in a "HTTP/1.1 400 Bad Request".

I think the 400 is because TSslHttpCli is not removing the fragment (#aria-atomic) when it is requesting the redirected URL.

Real browsers work fine.

Is this a bug?

On 4/10/2021 at 3:34 AM, Angus Robertson said:

I did wonder whether to write a check URL function, but it's not easy, except for space which is illegal in all URLs.  Lots of other special characters like /, & and % may be the result of previous encoding, so you don't know whether to encode them again.

 

But open to suggestions for a URL checker.

Well, you wouldn't be able to detect some issues but a URL/percent encoding checker that would check for invalid encoding like when there is % that is not followed by two hexadecimal digits, and of course if there is that invalid space character. Possibly also detect encoded control characters like %00-%1F. I may have already written something like this a while back (I have to check as I can't remember for sure) but I thought that a function like this might already be in ICS.

Not a big issue but perhaps something to consider.

13 hours ago, Lajos Juhász said:

The available information can be found at: http://docwiki.embarcadero.com/RADStudio/Sydney/en/Code_Insight_Reference

You can always try to enable logging and read the logfiles.

Thanks.... I'll start there and see if I can finally get it to work.

If anyone else has any other ideas or resources then please share.

Also, I'm still interested if it works for anyone here for any "real world" project.

Doesn't even work for me... it just says "CodeInsight: Stopped".

Does Code Insight even work for any C++ users with "real-world" projects?

Every time I update I hold out a little hope that Code Insight will start working again for my real-world C++ projects.

Unfortunately, this hasn't happened in years.

When I hit Ctrl+Space, it very briefly changes the mouse pointer but does nothing. In the "Projects" Tool Window, there's a "Code Insight Activity" bar. What the heck is that suppose to do? Mine seems to always show "CodeInsight: Stopped".

Is there a link that explains everything to check for in order to get Code Insight to work in C++? For example, all the requirements for it to work... and things to avoid that might break it. And it would be extremely useful if there was some way to see any messages that "Code Insight" might generating as to why it is failing.

Also, if anyone has a real-world C++ project and Code Insight actually works for you, then please let me know. I'm curious if it works for anything other than tiny demo apps.

7 hours ago, Angus Robertson said:

Yes, paths are meant to be encoded within the path delimiters, not / itself, unless after ?  So I did the simple fix of only handling space.  About 20 links failed my own tester with full encoding.  So a new version is in SVN.  That Canadian site also broke the ICS proxy due to not supporting absolute URLs used by proxies, that's been on my list to fix for a year, so got done as well. Testing is always useful, gets me to fix things. 

Thanks. I've updated and it fixes the issue I mentioned.

As you suggested, I'm also using onHeaderData to get the "raw" location URL (before encoding fixes/changes)... and basically throwing a warning if a location change URL contains a space character.

While I have your attention... is there a quick "ICS" way/function to check a URL to see if there are any encoding issues with it? Right now I'm only checking for space characters (very basic).

8 hours ago, Angus Robertson said:

Further to my last comment, I need to change the auto URL encoding for relocation so it does not process a URL that is already encoded correctly.  There will be another version soon.

 

Angus

 

Well, I'm not sure how you are handling this... but when there was a location change to "https://www.htmlvalidator.com/test/contains space.txt" it looks like it encoded the URL to "https://www.htmlvalidator.com/test%2Fcontains%20space.txt", so it fixed the space character but broke the '/' character.

Perhaps you should just change space characters to %20?

11 hours ago, Angus Robertson said:

I found the problem with HEAD and some redirections, it failed to start the redirected request until close was called, a bug that seems to have been there for many years.  Looked at my own link checker and I use GET not HEAD which is why I've never seen it.  Your 404 error page returns content even for 200 and HEAD.  It will be in SVN tonight.

 

Hello,

I downloaded the latest SVN and it looks like the major issue with the link checker stalling is fixed so thank you very much!

A minor issue... it looks like you're fixing the space character in location redirections which I think (in general) is a good idea but because I want my link checker to report issues and errors to the user, is there a way to detect that this fix/correction was done so I can report it to the user?

5 minutes ago, Angus Robertson said:

I found the problem with HEAD and some redirections, it failed to start the redirected request until close was called, a bug that seems to have been there for many years.  Looked at my own link checker and I use GET not HEAD which is why I've never seen it.  Your 404 error page returns content even for 200 and HEAD.  It will be in SVN tonight.

That's great... thanks!

Thanks for checking into this. My link checker should check for spaces in redirected URLs and report them and maybe fix them... I'll work on this, but this is not the main problem.

I am still confused why the RequestDone function is called twice? Shouldn't it be called only once?

The main problem is my link checker stalls out for some reason when this happens, and it never works again until the user exists and restarts my application. Did you find any indication of this issue also causing any serious problems or corruption with the operation of TSslHttpCli? I need to research more what is going on that this link stops/stalls my link checker.

Having the TSslHttpCli correct bad URLs like browsers do seems like a good idea in the cases where it is trivial... but if one is running a link checker like I am, I would want a way to disable these corrections or somehow know that they have been done so I can report the problem to the user.

On 4/7/2021 at 2:30 AM, Angus Robertson said:

Sorry, your redirection problem needs actual testing and not had time in the last two weeks for anything new. Angus

Thank you. Totally understand the time issue. However, this is a big issue for me so I hope that you will be able to find the time to help with it soon. UPDATE: Looks like you responded already... so nevermind... thanks!

Had this same problem when updating to 10.4.2.

Found this old file: C:\Windows\SysWOW64\RaizeComponentsVcl270.bpl

Deleted it. Tried installing again. Got a similar message but for different BPL.

Found this old file: C:\Windows\SysWOW64\RaizeComponentsVclDb270.bpl

Deleted it. Tried installing again. Now getting a "Cannot perform operation" dialog. Forcing shutdown. Reloading. Removing Konopka controls from "Install Packages".

Tried installing again. Success! Finally!

Now I bet it screwed up my 10.3 installation.

Thank you! I just updated to 8.65 and now 8.66 and 1.1.1k as I try to figure out this problem:

Unfortunately the problem persists in 8.66 with 1.1.1k and I'd still love to get that issue resolved.

I can say that 8.66 compiled fine (via Delphi environment) for my VCL usage with C++Builder in Sydney 10.4.1. Of course I had to set all the projects in D104InstallVclFmx.groupproj to generate all C++ files... I did not build the FMX projects though.

I use TSslHttpCli for link checking in my application. I just installed the latest recommended release, 8.65.

A user reported that the link checking in my application is stalling (getting stuck).

This URL seems to be causing a problem: 

http://www.ec.gc.ca/dd-sd/Default.asp?lang=En&n=C2844D2D-1

When I try to check it using TSslHttpCli and HttpCli1->HeadASync(), I get two calls to OnRequestDone.

1. First, there is a location change to "http://ec.gc.ca/Error 404.html" (could the space character in the new URL cause a problem?)

2. Next there is a first RequestDone call with StatusCode 0 and ReasonPhrase "", which I've programmed to ignore because I expect a 2nd call to RequestDone

3. Next there is a second RequestDone call with StatusCode 200 and ReasonPhrase "OK".

4. Something goes wrong here and my link checker stalls/stops and I'm not sure why but I'm wondering if something is messed up / corrupted with TSslHttpCli ... especially since the RequestDone function is being called twice (is this a bug)? Or perhaps I'm doing something wrong?

I'd really appreciate any insight into what might be happening. Thanks!