Angus Robertson

The ICS version of superobject now reports 'Error teDepth at line 1 column 6067 at depth 31' for the Json block, little more useful than nil result.  The information is all there internally, just not reported.   Not in SVN yet.

Angus

Thanks, I shall update the superobject  included with ICS appropriately. 

I did try the Json file on two web site parsers yesterday, one worked, one failed with an error but did not mention object depth, ran out of time to look further.

Angus

Did you see any downside to increasing SUPER_TOKENER_MAX_DEPTH. does it require more memory for every parse or something? 

Really need a way for tok.err to be exposed on failure, it's frustrating when superobject just fails for no obvious reason, although usually badly formatted json.

Angus

Look in the TIcsFtpMulti  component to see how it logs using the OnResponse and OnDisplayFile. events. 

As Francois suggests, you can build the OverbyteIcsXferTst sample that uses that component, check it yourself, then get your client to transfer the same file on their system with it, to see if it fails.  How large or small is this file?  The sample creates a log file.

Angus

Someone else mentioned a similar bug recently, but can not find the message. 

I fixed a bug calculating the speed of zero second FTP transfers 15 years ago that resulted in a similar error, but this would not happen with more recent releases.

ICS does have a n TicsLogger component that developers can add to their applications, but it is primarily for internal debugging of SSL applications, older components needed to be updated to log events and that was never done consistently.  Newer components like TIcsFtpMulti  and TSslHttpRest have ProgressEvent or OnHttpRestProg events,  and LogLevel or DebugLevel properties that can be used for screen display and file logging. 

If you continue using the older TFtpCli component you will need to add logging the onFtpClientDisplay and OnFtpResponse events to log the FTP protocol commands and responses so you can see at which point the error is happening.

Angus

If you mean you only want to accept LAN connections on port 80 or something, you can can do as you suggest, but this is irrelevant to SSL/TLS.   It is quite normal nowadays to respond with a redirection to the correct HTTPS  URL. 

Angus

Try building and running the sample OverbyteIcsMailQuTst.dpr in .\Samples\delphi\sslinternet\ and see if you get an error sending emails.  This is now the recommended way for sending SMTP emails, it tries multiple SMTP servers with attempts over several days to ensure email is sent.  

Angus

The main issue is you rarely want to put a lot of data into a memo, you can not view it while being received since it is updated so often, unless you only want to see the last few lines or are receiving data slowly, like alarm signals or something.  

The most efficient way to update a log window is to write data for display to a buffer (simple string is fine) and then use a timer in the application to empty the buffer to the TMemo every one to two seconds, which is about as often you can see it. 

That is exactly what more recent ICS samples do, like OverbyteIcsHttpRestTst1.pas, the AddLog proc builds the line and writes a file, the TimerLogTimer event updates the TMemo.   In some applications I simply discard most of the log lines if there are hundreds arriving each second.  

Angus

Most ICS applications do not use threads so synchronise is not needed,  However receiving data is blocked while you do anything in the OnDataAvailable event so not a good idea to update a memo if you are expecting to receive a lot of data. 

Angus

The CRYPTO_get_ex_new_index and similar functions have never been used by ICS.

Angus

IcsLogger output is intended for internal debugging of ICS components, not end user applications, and is always used with extra logging in the application.  So I have no idea what components you are using, with what IP addresses, ports or protocols, nor which of the numerous ways you have setup the module. 

If you have setup the module as an SSL TCP client, you need an SSL/TLS certificate for the ICS SSL server or it will not start.  It is more normal to set-up IOT modules as servers, so you contact them, but then they need a certificate.  

Suggest you read my earlier message again.

Angus

Rather than looking at OverbyteIcsSimpleSslServer and OverbyteIcsSimpleSslClient. which I assumed you wanted to talk to each other,. I suggest you look at OverbyteIcsIpStmLogTst instead, which can be configured as a server or client and handles all the SSL stuff for you, just setting SslCliSecurity or SslSrvSecurity as I mentioned above from combo boxes.  You should be able to talk to your Ethernet module with the demo. It does not support SSL client certificates, but you probably don't need them.

Angus

The declaration for sslSecLevel128bits has the comment 'RSA/DH keys=>3072, ECC=>256, FS forced, no TLS/1.0' which means you SSL/TLS certificate must have an RSA key length of 3,072 or later or EC-256.  You probably have a common RSA 2,048 bit certificate. 

In recent version of ICS with modern components, you generally don't set the OpenSSL security level, instead you set the client or server security level SslCliSecurity or SslSrvSecurity which set the ciphers, TLS version and security level for various scenarios.

Angus

PosEx was available in Delphi 2007, but became obsolete in XE3 when Pos got a third offset argument. 

function Pos(const SubStr, Str: UnicodeString; Offset: Integer = 1): Integer;

Abgus

The osslsigncode project is an alternative to signtool, I'd like to play with it, but I'm not a C developer.

Currently I have to enter a password for every single EXE or DLL I sign with the ICS open source certificate because the key is only a secure dongle, it's a pain. 

Angus

To save me looking at lots of old compilers, does anyone know in which version of Delphi that Pos got an extra argument to become equivalent to PosEx in D2007?  Hopefully D2009?

Angus

If you have already parsed the XML, and can compose the exact signed content, the ICS function IcsAsymVerifyDigest will verify with a private key and the hash digest, there are other function to create the digest with a private key and for HMAC signing with a shared secret. 

ICS will only work with specific OpenSSL versions it understands, and currently supports three major version, soon to be four when OpenSSL 3.0 enters beta next month.

You could probably embed the DLLs as a resource, unpack to tempdir and open them there, but I'm not planning anything like that.

There is an open source code signing project using OpenSSL https://github.com/mtrojnar/osslsigncode but it's 5,000 lines of C code and not trivial, Microsoft has made code signing quite complicated.  If anyone has built a Windows binary, I'd love to play with it.

Angus

I'd need to do more reading on XAdES to see what real cryptography is involved, but I've just finished updating the ICS Jose unit to handle signing and verification using JWK, JWS and JWT which involves hash digests, private and public RSA/EC keys and is used for REST APIs like Let's Encrypt, Google and Microsoft.  Anything using XML will be an older generation and should be easy to support in ICS, if there is a demand. 

Angus

Never heard of XAdES, but if it's a toolkit they should not be generating unwrapped base64 lines, I'd raise that as a bug with them on the basis that OpenSSL does not correctly support it...

Angus

Yes, I've only tried two of your four certificates, but they can be read by OpenSSL with short lines.  If none had worked with long lines, this would have been a lot easier to diagnose.

Angus

I made a mistake word wrapping one of the bad certificates and lost a character, when corrected it reads correctly with ICS and OpenSSL.

It is frustrating OpenSSL does not handle unwrapped certificates consistently and I've raised that as an issue.

But Michal can fix his original problem by ensuring the files are created according to the RFC with 64 character long lines, they are from some source other than ICS.   In theory I could word wrap them, but I think I'll just add a better error if OpenSSL fails. 

Angus

The original certificates are unwrapped base64 and that is how I tested them. however I manually line wrapped them for the OpenSSL mailing list since email does like 2,500 long lines. 

When I rest them line wrapped, my latest ICS says 'Reading X509 Base64 certificate: Error Cert 1 - error:09091064:PEM routines:PEM_read_bio_ex:bad base64 decode' for the bad certificate and the asn1parase command works for the OK certificate and gives a real error for the bad one.

I'll fix ICS to reject unwrapped certificates since OpenSSL can not reliably handle them. 

Angus

Another OpenSSL user has briefly tested the certificates and believes they contain badly formed UTF8, probably ANSI which is how this topic started.  The issue will be whether OpenSSL should open badly formatted certificates in future, even if it did in the past. 

Angus

No, don't need any more from you, OpenSSL should be able to read these certificates or say what is wrong with them, so it's their bug, not ours.

Angus

The certificates do not have private keys, but do have private names, they are client certificates.  No purpose in posting them here, but I will escalate to OpenSSL.  

Angus