Rollo62

Then you don't know me well, I make permanent, changes to all my core code libraries on a regular base Only I have too little sleep sometimes Even this change wouldn't be necessary, as it works right now and only degrades in performance under some few, rare conditions. But I'm willing to prepare for future devices, and to increase performance for my customers.

Of course I know, thanks for that nice piece of software Yes, TCP/UDP has very similar issues. Additionally in my configuration I cannot rely on delimiters, as I said, some use SOT/EOT delimiters, others use fixeld-length method. So if I only need to follow one protocol, that would be easy, but I need to follow several, different protocols. Yes, I could use a special handler for each protocol near the source, thats true. I already considered before, beside simple data types, to use a "class wrapper" für such tasks. Originally I thought about a wrapper for the simple types in the first place, but finally there could also linked lists, fifo or other classes be used. I have a working solution based on TBytes right now, and I look for fast/easy optimizations, so maybe I need to move through more than one step: 1. RawByteString 2. Linked list Since the 2. is a bigger efford to change at the moment, I would need to postpone that to a future change. But thanks for the suggestions anyway. I know that this is of course a very powerful method, transferring referenced types, which reduces memory copying a lot.

I have my land line connection for fast internet connection, TV, etc, and the land line phone is a kind of free "by-product". If I want that people don't call me, I give them my land line number

Sorry to confuse, I meant the original, old ASCII character set, of usuallly printable characters >= 32 < 127, pllus also including controls like Cr, Lf, Tab, ...). https://theasciicode.com.ar/ascii-printable-characters/space-ascii-code-32.html To me this is "ASCII 0 ... 127", sorry I have no better name for that set, maybe its compatible to CP_437. Since this kind of characters mostly do no harm and in memory these are 1:1 byte compatible. I see such character sets mainly produced from embedded devices, and like most that its a human readable "string".

Yes maybe, but then the list items will include incomplete chunks, and I have to torn and re-construct them later. When the source data arrives I have not enough time to analyse it, just push and go. If I push to lists, then that might happen ListMessages: - 11111 - 111 //<== Ok, this I can combine easily = 11111111 - 222 - 222222 - 2223333 // <== This I have to combine, torn apart, so that 3333 can be used later = 222222222222 +3333 ... - 33333333 = 333333333333 Yes, its possible, but a little more tricky.

Right, but as I explained I need to decouple source and processors by a circular buffer. The source data can provide data fastly, from a thread, while the processors will analyse and send the decoded data to different consumer. Thats why I'm looking for an intermediate buffer solution, to pre-process the data, until a complete chunk can be identified. I have sources were a transmission is not complete in one step, only the next transmission(s) may be complete one data chunk.

Yes, from Marco's guide this is exactly what I'm looking for, regarding the RawByteString If this RawByteString is still modern, then I will choose it as preferred solution. Since its that long in the Delphi environment maybe its very likely to stay there in the future too. I will try and check how it behaves and compares to the pure TBytes solution. Thanks to all for putting your arguments into this interesting discussion.

@Kryvich Thanks, but its from 2008. Is this still 100% valid, after all that FMX, mobile, ARC, .... additions ?

Look for example into the Bluetooth LE communication functions.

See my first post

@David Heffernan Sorry to be unclear with that, I wrote: Maybe I should say that it contains maybe 95% string and 5% binary data, as it may come from various sources. Mainly printable ASCI ( 0 ... 127) fitting in one Byte, but as I said also pure binary ( 0 .255) can occur from some providers. How are the strings delimited in the buffer? Usually SOT with CrLf or Lf as EOT for the ASCII data representation, the binary data representation also has SOT/EOT of some kind but can be fixed length data too. Yes, the data will be distributed into various processors, which know how to handle the data right. But I need a common buffer type for the intermediate buffer, which distributes the data. TBytes is preferred type for manipulating binary data. Absolutely, unless I have a mixture of both. I have not really checked yet in detail what is the internal difference between RawByteString and AnsiString internally, I assume this are mainly the codepage parts. Right, thats my goal too. I try first to make custom types, derived from standard classes, with some disadvantages. Maybe its also worth to wrap those standard classes into a new wrapper class or even generics, opening full featured class handling, but I wanted to keep such stuff more low level, and not to overdue. There should be something already fitting this case, from the above candidates. As always I'm open to all non-standard proposals, no reason to take out tar and feathers here, but I know that this proposal is seen as bad practice. In case it works and solves my issues, I could live with that, but still there might be better solutions around. I'm with you too, since already the name proposes what purpose it has. If not only the documents would say "dont use it". Well thanks, was not aware of that important feature. Right, thats what I hope too. Makes live easier, even if the purists might say "binary its not a string". So maybe I should dig deeper into a RawByteString implementation now, and check if that might fit well. Would be good to know the future of RawByteString, are there plans to phase this out ?

I do. I think even a simple assignment operator is a showstopper ....

I tend to see the right candidate would be AnsiString, as it supports codepages for maybe future use, but the support in Delphi of AnsiString I also have in question. Shall it stay, or shall it go ? ( according to a well known song )

No, just basic ASCII 0 ... 127, but I cannot prevent that in future there maybe some "failure" in the data. We work with several vendors for the data sources, so there might be some other codepage inbetween. But I consider this is a failure now, and need to find a workaround once this happens.

Yes I have that, but the strings are so much more elegant. I would whish to have better TBytes and dyn. array support like that in Delphi.

As I said 95% of data is string source, what decision to make then ? I would tend to prefer string over Bytes, as this is the main data used.

The problem is the speed. Incoming data can be very fast, so I need to store them in a ringbuffer, and need to extract and process later. Excactly for that reason I'm looking for the right buffer data structure. Since the data is maybe 95% string and 5% binary, my thought is that string-like could be preferred.

@Kryvich Right, that was my first thought too, but then see the docs ... I don't want to change my code all the time in the future. By the way, there was a similar discussion here, regarding the codepare 437. I found another possible reason to use codepage 437, because the TEncoding includes something like this: class function TEncoding.GetEncoding(CodePage: Integer): TEncoding; begin case CodePage of {$IFDEF ANDROID} 437: Result := TCP437Encoding.Create; {$ENDIF ANDROID} 1200: Result := TUnicodeEncoding.Create; 1201: Result := TBigEndianUnicodeEncoding.Create; CP_UTF7: Result := TUTF7Encoding.Create; CP_UTF8: Result := TUTF8Encoding.Create; else Result := TMBCSEncoding.Create(CodePage); end; end; So I need Android, which means codepage 437 will be supported in a special way, maybe with better performance, to convert back and forth with Unicode strings. Anyway, @Remy Lebeau proposed to use "codepage 28591 (ISO-8859-1) instead", which probably should be fine on Android too. He also notes that RawByteString may have conversion issues Since my data mainly includes "string data", and only in some cases "binary data", I think early conversion to string is better for manipulation, than carrying and manipulating in TBytes, and only late convert to strings. Regarding RawByteString, this has the ugly note in the docs, but AnsiString seems to be returned as a full citizen in Delphi again, so maybe AnsiString with codepage 28591 or codepage 437 is the way to go ?

@David Heffernan I basically thought the same, only a feeling that TBytes gets a little slower. Not performance tested yet, but the data comes from external device. The reason I like strings if because they support chopping, deleting unwanted chars, recombining them very effective. And thats what I need with the data, I receive it into a ringbuffer fastly, then need to analyse, chop and redistribute into many other places. The problem is that the data may come from very different external devices, but need to be manages all in the same processor. You're right, thats why I also consider AnsiString, but is this still recommended to be used ?

Hi there, I'm looking for some communication with the Alexa, and a custom skill. The idea is that Alexa uses a AWS Lambda function, and may control a web-service to activate some stuff. Thats usually the easy way, via Alexa Intents: A.) Normal command flow So far so good, the processing delay is usually 1-2 sec., so I still would call it real-time. B.) Permanent command flow What I need is to use unintended data transmissions from web-service. I could see to ways to achive that: 1. Register some permanent callback: 2. Keep the AWS session open Either way: Alexa seems to be a one-way, as A.), I have not found too many info about using Alexa as a kind of TTS-output. If 1.) or 2.) is possible at all, I cannot say, maybe there is even another option I don't see yet. Like triggering any voice output by web-hook, or the like, that would be perfect. Of coarse the application shall use Delphi/FMX to control finally, but the loophole seems to be the limited Alexa environment. Maybe somebody has a helpful hint, or a good solution, to push me in the right direction ?

Thanks I was not aware of this. But the documentation is somewhat hidden, here I found some older info. @Dave Nottage Not sure what you need exactly for "simulation", if you mean the remote access then the above recommendations are fine. Maybe you only want to simulate some test-cases, to check and proof your application code and functionality. What I use sometimes is a somewhat brute method: Usually I have a serial component that I know and trust well, so I know its functionality, or I can test this separately in my office. There should be a callback somewhere, which maybe fired in a thread, like EvOnRead and a function for a command to write. Sometimes, if the connection is not very critical its OK for me to build a mockup around those core Read/Write functions. Of coarse timings, baudrate, flow.control, etc. is a little more difficult or even impossible to simulate, but in the end those callbacks will present the data to me. I can make stress-tests with the hardware callbacks, to simulate chunked frames, timeouts, etc. For for a basic simulation of read/write flow for the data, such mockup helped me many times. With that approach I'm even able to write a simulation page, in the very same application, send/receive specific commands, and see how the application reacts to it. I know its a little crude, but its just an idea. Maybe this might be good enough for your case too. Of course there will be the day when you need to move to the real hardware ...

To avoid such hard-to-find mistakes, I use proper begin-end scopes ALWAYS ...... ( and I really, really mean ALWAYS, all the time ). That needs a lot of discipline over the years, that I dont always have. <cross-fingers> Since I did it that way, I never saw such failures again. </cross-fingers>

Maybe thats helpful too, from Stephen Ball's nice video lectures.

https://delphidabbler.com/notes/version-numbers.html

I like inline vars, but don't see much importance for my right now. Thats why I don't use them (and also for backwards compatibility reasons) until they are really stable and tested 150%. Same I like ternary operators, but in Delphi we only have the lame "IfThen( ..." replacments. I would like to see them in the language too, which will help a lot to shorten some specific code. Its a higher importance to me, but still not a killer feature. There are more important things to do, like closing current issues, before opening new, un-needed issues.