Jump to content
Der schöne Günther

Cast an array of numbers to array of bytes without re-allocating

Recommended Posts

Posted (edited)

I have some API that allocates and returns a TArray<Word>.

I would then like to process this array with a library that operates exclusively on TArray<Byte>.

 

How can I cast my_word_array to my_byte_array without re-allocating space for a new array and then copying the entire content over? 

 

My only idea so far was a clunky workaround using the absolute directive and manually overwriting the array's length indicator in memory:

procedure changeLengthIndicator(const firstElement; const newLength: NativeInt);
var
    ptr: PNativeInt;
begin
    // Source regarding memory layout:
    // https://docwiki.embarcadero.com/RADStudio/Alexandria/en/Internal_Data_Formats_(Delphi)#Dynamic_Array_Types

    ptr := Addr(firstElement);
    Dec(ptr); //set pointer to length indicator
    ptr^ := newLength;
end;

procedure p();
var
    words: TArray<Word>;
    bytes: TArray<Byte> absolute words;
    byteCount: NativeInt;
begin
    words := [$0FF0, $FEFE, $0000, $0001, $ABAB];

    byteCount := Length(words) * SizeOf(Word);
    changeLengthIndicator(bytes[0], byteCount);

    processBytes(bytes);
end;

Is there a better option?

Unfortunately, there is no making sure no one will use the words array after the call to changeLengthIndicator(..).

Edited by Der schöne Günther

Share this post


Link to post

Ok, what does then

SetLength(words, Length(Words)*2);

SetLength(TArray<Byte>(words), Length(Words) div 2);

with the array? 😉

 

Btw, it will also reallocate and copy the memory if needed so forget SetLength().

Does your version leak memory?

 

Share this post


Link to post

I see nothing wrong with using absolute in this case, however you could make it a tiny bit stronger by placing it all in a class and hiding the storage.  Maybe use TList as a starting point, but add accessor methods to fetch/append/insert/delete for each type. Keep a state variable so you know what mode its in and only have to switch modes when you change.  Also, keep counts for each supported type so you can set the length appropriately.  The list would always be an array of integers behind the scenes, but it then could contain an odd number of bytes or words.  A ForEachByte or ForEachWord routine could be coded to use a stack to set and reset the state variable if necessary.

 

I have done something similar with absolute when I needed to compare large blocks of data.  An integer compares or moves at just short of 4x as fast as looping through 4 bytes.  Disclaimer: This doesn't mean you should use this technique because you can. Its good practice to always measure to find the real bottlenecks and not add complexity to optimize when the net result is still nearly identical.  

  • Thanks 1

Share this post


Link to post
54 minutes ago, Steven Kamradt said:

Keep a state variable so you know what mode its in and only have to switch modes when you change.

That's actually an interesting idea. It will store if it's in "byte" or "word" mode, and if you're trying to get the "wrong" data, it will instead throw a runtime exception. That's certainly better than what I have right now.

Share this post


Link to post
Posted (edited)

Just remember such hacks could break if compiled with RTL that has another layout. Besides this, the solution is quite OK. However, if at least one of 3rd parties you use accepted raw pointers, the issue wouldn't exist at all.

Edited by Fr0sT.Brutal
  • Thanks 1

Share this post


Link to post
1 hour ago, Fr0sT.Brutal said:

such hacks could break if compiled with RTL that has another layout

Of course. That's what unit tests are for 😎

 

1 hour ago, Fr0sT.Brutal said:

if at least one of 3rd parties you use accepted raw pointers, the issue wouldn't exist at all

True. Or, even better and less c-like, IEnumerable<Byte>

Share this post


Link to post
15 hours ago, Der schöne Günther said:

Of course. That's what unit tests are for 😎

Fair point

15 hours ago, Der schöne Günther said:

Or, even better and less c-like, IEnumerable<Byte>

I'm a bit old-school. Pointer+length work with almost any language, interfaces are mess (especially Delphi ones)

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×