Any Known Issues with ZCompressStream?

[Steve Maughan 26]

18 hours ago, David Heffernan said:

I can't understand why you would. Aren't you likely just to end up changing your code for no reason, given that the defect is almost certainly not in your compression library?

I won't change my library but I'd still like to be aware of any newer, and maybe better, libraries. Having had a quick look it seems to be third party. I had thought it was shipped with Delphi since it was System.Zip2. I try to minimize the use of third party components so definitely won't switch, or even look any further.

[A.M. Hoornweg 144]

2 minutes ago, Anders Melander said:

That is not the purpose of the checksum. The purpose is to guarantee that the output is correct.

Which brings us back to the solution I proposed originally. Ensure that the compressed data is valid (with another checksum) before trying to feed it to the decompressor.

[Vandrovnik 214]

26 minutes ago, A.M. Hoornweg said:

Which brings us back to the solution I proposed originally. Ensure that the compressed data is valid (with another checksum) before trying to feed it to the decompressor.

I still believe that decompressor with wrong input should not enter an endless loop.

If you have 2 bytes checksum, you still have a 1 : 65536 probability, that it will not detect wrong input.

With 4 bytes checksum, it is 1 : 256^4, much better, but still not 100 % safe.

[Fr0sT.Brutal 900]

1 hour ago, Steve Maughan said:

I won't change my library but I'd still like to be aware of any newer, and maybe better, libraries. Having had a quick look it seems to be third party. I had thought it was shipped with Delphi since it was System.Zip2. I try to minimize the use of third party components so definitely won't switch, or even look any further.

The core of zlib is borrowed from C sources compiled to obj so you can build your own obj from recent version and try to use them.

Anyway I think the problem MUST be reported to QC together with problem files because the issue is critical, it could cause DoS on any Delphi server.

 

10.3 sources say they use zlib 1.2.11 which seems to be the latest version

Edited September 30, 2020 by Fr0sT.Brutal

[Anders Melander 1782]

5 hours ago, Vandrovnik said:

I still believe that decompressor with wrong input should not enter an endless loop.

Yes, that's probably a bug.

I don't know if ZLib is supposed to be able to handle anything you throw at it gracefully but being a general purpose library I would think so.

 

We haven't seen the code that exhibits the problem or a sample of the data that causes it, so I'm hesitant to declare the problem a bug in ZLib or the Delphi wrapper.

 

4 hours ago, Fr0sT.Brutal said:

Anyway I think the problem MUST be reported to QC together with problem files because the issue is critical, it could cause DoS on any Delphi server.

If it's caused by a bug in ZLib then it would be a good idea to also report it at the ZLib repository: https://github.com/madler/zlib

I don't think Embarcadero is likely to fix bugs in ZLib.

[Fr0sT.Brutal 900]

14 hours ago, Anders Melander said:

If it's caused by a bug in ZLib then it would be a good idea to also report it at the ZLib repository: https://github.com/madler/zlib

I don't think Embarcadero is likely to fix bugs in ZLib.

Sure. But it could be a bug in RTL's wrapper (there are chances for this because zlib is used much more widely so such a critical bug would've been reported much earlier).

[Anders Melander 1782]

4 minutes ago, Fr0sT.Brutal said:

But it could be a bug in RTL's wrapper

Yes it could. And before reporting it as such one should find out if it is.

 

5 minutes ago, Fr0sT.Brutal said:

zlib is used much more widely so such a critical bug would've been reported much earlier

That's not really a reliable metric to determine where a bug is located. Regardless, these kind of problems are regularly found in ZLib. Just try to search for "zlib loop" or go through the issue list at Github.

[User-69 0]

Today I investigated similar topic.

I figured out that zlib writes the adler32 checksum to the file, but the implementation never checks them. So any flipped bit causes such issues.
I did not found an easy way to implement the error check as you need to do that in zLib.pas.

 

Here is explaind where to find the checksum in the file  (ist simply msbfirst, so you can easyly read thet strait forward). only in german.

https://www.gocher.me/GZIP

The current adler32 checksum can be get while debugging.