Jump to content

David Heffernan

Members
 View Profile  See their activity

  • Content Count

    3711

  • Joined

  • Last visited

  • Days Won

    185

Posts posted by David Heffernan

  3. Posted

    7 minutes ago, Rollo62 said:

    I tend to see the right candidate would be AnsiString, as it supports codepages for maybe future use,
    but the support in Delphi of AnsiString I also have in question.

    Shall it stay, or shall it go ?  ( according to a well known song )

    At the start of this thread you said that the data was binary. Now you say it is ASCII. Hard to give advice on this basis.

  4. Posted

    Would be perverse to use 16 bit Char to store 8 bit data. In terms of performance byte strings and byte arrays are similar but if anything byte arrays will be faster. Precisely because they don't have coy on write. No idea why you thing strings perform better. 

     

    My guess is that your antipathy to byte arrays is a hangover from the legacy Delphi anti pattern that byte arrays are handled as strings. 

  7. Posted

    3 hours ago, Steve Maughan said:

    Thanks — I wasn't aware of System.ZIP2. I'll take a look

    I can't understand why you would. Aren't you likely just to end up changing your code for no reason, given that the defect is almost certainly not in your compression library?

  8. Posted

    1 minute ago, aehimself said:

    Ummm... I did not? I just offered a free to use alternative.

    It's pretty bad advice. Changing algorithm and implementation without any justification or rationale. Seems like you are advocating trying libraries at random. If every time you encounter an issue you replace the lirbsry, after a while you'll have run out of libraries. 

  9. Posted · Edited by David Heffernan

    2 hours ago, Steve Maughan said:

    I thought these were just a wrapper around standard ZLib routines

    They are. 

     

    2 hours ago, Steve Maughan said:

    It could have been corrupted after being saved.

    That's the a very plausible explanation. File corruption is something that does happen. 

     

    You'll want to reproduce the issue before trying to solve the problem. And if it is file corruption then the solution is somebody else's problem.  

    • Like 1

  10. Posted

    Just now, aehimself said:

    If you don't need anything fancy, you can use System.ZIP (or the updated System.ZIP2, which is a drop-in replacement offering some extras). I'm using Zip2 with smaller modifications, works like a charm.

    How did you diagnose that the defect was in ZCompressStream or ZCompressStream? 

  15. Posted · Edited by David Heffernan

    Imagine if you have users with names that don't begin with one of the 26 letters used in the English language? 

     

    What you should do is abandon this UI approach and let the user type. 

    • Like 3

  19. Posted

    There's a lot of noise in here. It seems you don't really understand where these characters are coming from and are in trial and error programming mode.

     

    The advice from the wise heads here is to understand what is going on, and then work out how to tackle it. 

     

    You don't seem to want to heed that advice. That's fine, it's your choice.  But we don't need a blow by blow account of your trial and error coding. That's only meaningful to you. 

    • Like 1

  21. Posted · Edited by David Heffernan

    1 hour ago, Mike Torrettinni said:

    I was referring to variable iterating For..in loop - in this case full array record content is copied into variable. No? At least that's what I see from the simple example above.

    Yes that is correct. My argument stands.

     

    Copying the full record into a local is only expensive (compared to reading a value from the record in-situ in the array) if the record is large.  For a small record, copying a handful of bytes costs no more than reading even a single byte.

     

    I'm arguing against your claim that there would be a performance hit using a for in loop even for small records.  For large records there will be a hit.  Not for small records.

  22. Posted

    40 minutes ago, Mike Torrettinni said:

    Iterating a few large records (390MB) or lots of small records (10000x 0.039MB), could have similar performance effects.

    No. Because when you iterate over an array, you typically don't read all of the content of each item.

     

    Imagine that all you do is look inside each record for an integer ID. If the record is huge, then you can just read a single integer, and move on, if using a classic for loop with an array of record. But if you use a for in loop you have to copy the entire record before reading the single integer ID. That's wasteful.

     

    In the case of a smaller record, let's say small enough to fit into a cache line, then reading an integer from the record has the essentially same cost as copying the entire record and then picking out the integer.

     

    So the trade off depends hugely on the size of the record, and what proportion of it you actually need to access.

    • Like 1

  23. Posted

    25 minutes ago, borni69 said:

    I agree with you...

     

    And after this back and forward discussion in this thread I think I have a solution.

     

    I will only send out characters  and commands that are handled by TjsonString.create() 

     

    So I think my code will be something like this  

     

    PS:  there is  0b 0b  before E-K in rawText

      

    
 rawText := 'E-K ble æøå  Test // 98';
  ajson := TjsonObject.Create;
  try
   ajson.AddPair('text',TJSONString.Create(rawText));
   RawtextOut := ajson.tostring;
  finally
    ajson.Free;
  end;

  textOut:='';
  for ch in RawtextOut  do
  begin
   if (ch >= #32)  then
   textOut := textOut+ch

  end;

  memo1.Lines.Text := textOut;

     

    result

    {"text":"E-K ble æøå  Test \/\/ 98"}  characters I dont want are removed...

     

    also line break tabs will be handled correct  ..    \r\n

     

    Thanks for all your help..

     

    B

     

     

     

     

     

    Better hope that there are no line breaks .....

  24. Posted

    14 minutes ago, Mike Torrettinni said:

    or smaller records but you have a large number of them, so iteration copies smaller memory but a lot of times

    Why would there be an issue for small records? Presumably you are iterating over the array because you want to look at the content of the record. For a small enough record, there won't be any difference in perf between reading a field and copying the entire record.

  25. Posted

    50 minutes ago, borni69 said:

    Sometimes they copy text from word / email  etc, and then we get characters we dont want..  I am not sure what they are, we like to keep linbreak tabs ect, but not this character showing as a ?  or a as seen in above image.

     

    Your problem is not how to remove characters, it is to work out what characters are to be removed. As is so often the case, the hardest part of any programming tasks is determining the correct specification.

×