Tommi Prami

https://quality.embarcadero.com/browse/RSP-39546?filter=18027 Vote and/or collaborate if feels serious to you... -Tee-

Even tough this is very weird example, but seems like something that with some bad luck one could get hit by that. -Tee-

Please report to the Bug Report that you could reproduce it...

RTL has Buffered file stream now, but I think there is no buffered Memory stream, I think. is there fast implementation available? I made long long time ago one, but that was not super good implementation and don't have code anymore. At least then it made some code way faster, and Stringbuilder I believe is not super fast, and and can't handle binary data 🙂 -Tee-

Not problem for me, but "interesting" choice to implement those features with helper, they kind of can modify existing classes quite freely if they want 🙂 -Tee-

To me quite the opposite, just the release we needed. Quality updates should come more frequently IMHO. -Tee-

Fellow forum members educated me of feature I did not know existed, or I knew, but I never used it this way. I've attached pdf-file for the info. Thanks all helping to get the call count, this is pretty cool. Get call count with Delphi Debugger.pdf

Now that went through TMemoryStream code, it is buffered... I must confuse it into some other stream class, sorry about that. But really interested how to get call count of the non breaking breakpoint tough...

That would be good to investigate, how I can see the count like that, never knew you could setup a breakpoint like that, so how to do that?? Non braking breakpoint yes, but how to see how many times it is called... Also should run as release build with optimizations on, to see how it runs then. (This was pretty much similar, little bit slower, but OK)

I do fast and crude test. Good that I did, speed gain is nice, prosentually, but it is much faster that I though: procedure TForm3.Button1Click(Sender: TObject); const LOOP_MAX: Integer = 90000000; DATA: RawByteString = '0123456789'; var LStopWatch: TStopWatch; I: Integer; LStream: TMemoryStream; LDataLength: Integer; begin LDataLength := Length(DATA); LStream := TMemoryStream.Create; LStopWatch := TStopWatch.StartNew; try LStopWatch := TStopWatch.StartNew; for I := 1 to LOOP_MAX do LStream.Write(DATA[1], LDataLength); LStopWatch.Stop; finally LStream.Free; end; Memo1.Lines.Add('No preallocation: ' + FormatFloat('0.000s', LStopWatch.Elapsed.TotalSeconds)); LStream := TMemoryStream.Create; LStream.Size := (LOOP_MAX * LDataLength) + 1000; LStopWatch := TStopWatch.StartNew; try LStopWatch := TStopWatch.StartNew; for I := 1 to LOOP_MAX do LStream.Write(DATA[1], LDataLength); LStopWatch.Stop; finally LStream.Free; end; Memo1.Lines.Add('Preallocation: ' + FormatFloat('0.000s', LStopWatch.Elapsed.TotalSeconds)); end; No preallocation: 1,150s Preallocation: 0,647s Nice speedup, prosentually, but time saved in real world situation is very small. So no point of optimizing this, in my current case. This way. Maybe my mary failed me, and my original optimizations where pre FastMM times. or I just looked how many percent faster it was 😉 Or both.... -Tee-

For sure. My experience is that if you write lot of stuff to to stream, and if there is lot of reallocation, it'll be quite slow. Need to measure this for sure, before commit into anything. All I know i've used buffered stream to speed up very close to same situation now. I just remember that some parts of the code got way faster, and overall saw significant speedup. I think this comes down to how much data is written and how small pieces and so on, in other words, how many reallocations will happen in real world. I think I logged the final stream sizes for a day (it was an server which made Xml files) and used Stetson-Harris method to pic some nice size for initial allocation size, and adjusted growth strategy to get some kind of balance and not to allocate way too large buffers. Concatenating strings on FastMM is fast, but still I managed to make nice speedup back then. It is quite easy to make Buffered stream, but testing and handling all possible corner cases you first miss... That is why I am asking if there is good implementation that I could test, does it make sense or not (in this case) 🙂 -Tee-

Reallocation for sure.

Hello, QP here: https://quality.embarcadero.com/browse/RSP-38978 If you care, vote and or comment. -Tee-

That sounds smart implementation, if blocks are big enough, SaveToFile etc (combining the data to one, or reading it to elsewhere) should be pretty fast also.

As far as I know TMemoryStream does not buffer, at least in a way that I want to use it. -Tee-

Current code writes stuff into the memory stream in small chunks, it would be _easy_ optimization to have buffered memory stream. I used to have very crude memory stream that allocated memory with some mechanism that I can't remember, it was mainly used to write XML data back then, but it made some code 20x faster. (if I recall), but some isolated cases where way faster, than reallocating all the time. One thing I remeber is that implementation did reallocate always when buffer got full, so was not super smart, just some simple growth scheme. Sure I could rewrite the code at first place, but sometimes it is legacy code of some third party library, so rewriting might be too big of a task. Would be better to get some maintained library instead, if available. But it always is case by case call which way to go. -Tee-

Sweet, could you publish code to QP as a starting point, and/or report your findings there? -Tee-

I was pondering that if some one has good list of exceptions to catch and what these exactly mean, it would help, so no need to go through the code to find out. I mean that if I have no something like on E: Exception do ... I would prefer something like that (obviously just made up an example) on E:EIdSomethingSomethins do // Internet is most likely down on E:EIdTimeout do // Server did not respond, is could not connect on E :IdReadTimeOut do // Connected but could not read data ... Else Raise etc.. So I could more precisely catch all internet related exceptions and give more exact error messages. I've done something like that in the past but it was more than less cumulated info over time and not always very well analyzed, I think. I bet if someone have good exception handler and post here, and have some peer review and suggestions, we could have good starting point for all that want to do something similar

Giving own message, or having certain flow. Exception messages usually are not super good to show for users, because they are bit too technical. That's all. -Tee-

This library is widely used, author uses it in his own work so it is not dead. If main developer is busy at the moment, others could chip in. I think he has time to merge solid pull requests. Also if the open bug iare no something that is deal breaker for you, just dive in.

I am pretty sure it can't be (RAW) binary (will most certainly contain characters that are illegal in XML file) . Most likely Base64 encoded. But should check from Server side developers/docs

https://nakedsecurity.sophos.com/2022/03/29/zlib-data-compressor-fixes-17-year-old-security-bug-patch-errr-now/ -Tee-

Bug report link please, quite useless to report only here, need to report to Embarcadero. And link so we can vote.

Bug report link please, quite useless to report only here, need to report to Embarcadero. And link so we can vote.

Bug report link please, quite useless to report only here, need to report to Embarcadero. And link so we can vote.