David Heffernan

We've all done it. It never works out.

There's a lot of noise in here. It seems you don't really understand where these characters are coming from and are in trial and error programming mode. The advice from the wise heads here is to understand what is going on, and then work out how to tackle it. You don't seem to want to heed that advice. That's fine, it's your choice. But we don't need a blow by blow account of your trial and error coding. That's only meaningful to you.

REMOVED, sorry, was dupe

Yes that is correct. My argument stands. Copying the full record into a local is only expensive (compared to reading a value from the record in-situ in the array) if the record is large. For a small record, copying a handful of bytes costs no more than reading even a single byte. I'm arguing against your claim that there would be a performance hit using a for in loop even for small records. For large records there will be a hit. Not for small records.

No. Because when you iterate over an array, you typically don't read all of the content of each item. Imagine that all you do is look inside each record for an integer ID. If the record is huge, then you can just read a single integer, and move on, if using a classic for loop with an array of record. But if you use a for in loop you have to copy the entire record before reading the single integer ID. That's wasteful. In the case of a smaller record, let's say small enough to fit into a cache line, then reading an integer from the record has the essentially same cost as copying the entire record and then picking out the integer. So the trade off depends hugely on the size of the record, and what proportion of it you actually need to access.

Better hope that there are no line breaks .....

Why would there be an issue for small records? Presumably you are iterating over the array because you want to look at the content of the record. For a small enough record, there won't be any difference in perf between reading a field and copying the entire record.

Your problem is not how to remove characters, it is to work out what characters are to be removed. As is so often the case, the hardest part of any programming tasks is determining the correct specification.

The hardware is designed to handle indexed access efficiently. I've seen plenty of cases where incrementing a pointer inside the loop is slower. After all, you have an extra variable that needs to be incremented. Hard to see why that would be faster. No. The memory access pattern is identical. It's just a sequential pass across the array.

The two 0b characters are valid UTF-8 characters. They are this character: https://www.fileformat.info/info/unicode/char/000b/index.htm What you need to do is decide exactly what your specification is. The question you have asked doesn't seem to match the example you have provided.

Not necessarily. And in any case, if you wanted performance, you'd not have chosen Delphi.

This is why I always implement my enumerators as records so that they can be automatically allocated. Will it? Faster than array indexing? That seems to be something of a myth in my experience.

Correct.

Loop variables are assigned using assignment semantics. That's all you need to know.

It's the same issue for both types of for loops. The value of the loop variable is undefined if the for loop terminates normally. In your case that corresponds to the condition never having been met. Unfortunately though, whilst you are aware that there is an issue here, you have not correctly understood what the issue is. That reasoning is incorrect. The loop variable i does not change in case of a normal termination, e.g. the break in your code. Before you consider for in loops I recommend that you correct your misunderstanding of for loops. Executive summary. There is an issue with for loops of both kind. It is the same issue for both kind of for loop. But the issue is not what you have described.

Define what you mean by a non UTF8 character, remembering that this is a variable width encoding.

Why are you cross posting? https://stackoverflow.com/questions/64010603/delphi-getting-stream-read-error-on-really-simple-program At every site where you cross post, potentially different people will provide you the answer. Which means that more people than necessary spend time on this. As a broad rule, people who provide this sort of help don't appreciate this kind of behaviour.

DELETED: because it was nonsense

This is the correct way to do this. It works fine. The problem is elsewhere. Don't create and raise a new exception.

Packing does make a difference here. It may not change the layout, but it changes the alignment of the type.

No it shouldn't be declared as packed. The structure, as is invariably the case for Windows structures, should be aligned.

Nobody can diagnose the problem without seeing your code. Once you show us a minimal complete reproduction then it will be straightforward to explain the behaviour of your program.

That trade off depends on the size of the record. For small records then performance is better if the items are stored directly in a contiguous array.

OK, this is making more sense to me, and I can better appreciate the inconvenience.

Maybe. I guess I might not be encountering the problem regularly. You must be facing it very regularly I guess. Isn't that a different issue?