Better TStringList in Spring4D or elsewhere

[Dave Novo 57]

Hello,

 

Amazingly, after so many years, Delphi still does not have an effective TStrings implementation of InsertRange (insert a range of strings at a specified index), DeleteRange (delete a range of entries at a specific index). Of course IList<string> in Spring4D has these, but then you lose many of the helpful things that TStrings does have (like .CommaText, .Text, BOM handling and others).

 

I see that Spring4D has a TStringsAdapter and implements many of the missing methods, but does so following the TStrings way (i.e. implements insertRange by calling FStrings.Insert over and  over in a loop which is moving the memory around many times). Does anyone know of a TStringList replacement that keeps all the features of TStrings but adds more robust insertion, deletion etc in an efficient manner?

Edited Monday at 10:34 PM by Dave Novo

[Anders Melander 2073]

28 minutes ago, Dave Novo said:

Does anyone know of a TStringList replacement that keeps all the features of TStrings but adds more robust insertion, deletion etc in an efficient manner?

Robust? It seems to me that the current implementation is pretty robust.

 

Just because it doesn't do or behave as you'd like doesn't make it wrong; It's an age-old utility class that has to conform to a certain contract in order to not break a gazzilion applications.

 

Regardless, it should be pretty easy to implement a helper that hacks access to the TStringList.FList array and then simply manipulate that array directly. Or you could just copy the TStringList source and modify it to your liking.

[Stefan Glienke 2150]

InsertRange and DeleteRange, as you describe them, are not possible without changing the existing TStringList behavior regarding OnChanging/OnChange - currently, if you insert or remove several items in a loop using Insert or Delete, on every OnChanging/OnChange call, the TStringList has the exact Count and Items before/after that one insertion/deletion. With a bulk insert or deletion, that would not be the case.

 

Also, fwiw, the adapter in Spring is to wrap an existing TStrings as an IList<string> if some API requires - not to make TStringList any more performant. Regardless of all the nifty features of TStringList for most string-only operations that need to be super performant, I would rather avoid using it because its internal storage layout is terribly wasteful (i.e., storing those string/object pairs).

 

Also, that adapter is for TStrings and not just for TStringList.

Edited Monday at 10:41 PM by Stefan Glienke

[Dave Novo 57]

Hi Anders,

18 minutes ago, Anders Melander said:

Regardless, it should be pretty easy to implement a helper that hacks access to the TStringList.FList array and then simply manipulate that array directly. Or you could just copy the TStringList source and modify it to your liking.

Of course I can hack the FList, hack the Capacity, hack the count, hack the Notify and and/or reimplement the entire StringList. Just wondering if anyone has done so already.

[Dave Novo 57]

2 minutes ago, Stefan Glienke said:

InsertRange and DeleteRange, as you describe them, are not possible without changing the existing TStrings behavior regarding OnChange - currently, if you insert or remove several items in a loop using Insert or Delete, on every OnChange call the TStrings has the exact Count and Items after that one insertion/deletion. With a bulk insert or deletion, that would not be the case.

Of course, it is one of the problems with the current TStrings. It was simply not designed for efficient bulk operations. For example, as long as I was notified which items were going to be removed/extracted/inserted either before or after the actual  removal/extraction/insertion then I could handle whatever I needed to do. I dont need to be notified one item at a time. 

 

I am mainly wondering if in the last 30 years anyone had written a robust implementation that handled these bulk operations. I could spend a few hours/day creating something based on IList<string> and copy/pasting relevant code from TStringList to handle the missing/useful functionality but again, just wondering if anyone did so already.

[pyscripter 829]

1 hour ago, Stefan Glienke said:

InsertRange and DeleteRange, as you describe them, are not possible without changing the existing TStringList behavior regarding OnChanging/OnChange - currently, if you insert or remove several items in a loop using Insert or Delete, on every OnChanging/OnChange call, the TStringList has the exact Count and Items before/after that one insertion/deletion. With a bulk insert or deletion, that would not be the case

This is not quite true. TStrings does not define Insert/DeleteStrings but it has an AddStrings function.   This function is inefficient (adds the strings one-by-one), but the operation is enclosed in Begin/EndUpdate, so you only get change notifications before and after the whole operation.  In an analogous way, an InsertStrings function would/should only notify before and after the whole operation.   There is no need/requirement for calling Changing/Changed per item.

 

@Dave Novo  It is indeed easy to write a TStringList class helper that adds efficient implementations of Insert/DeleteStrings.  You can get access to the private  variable FList using the well known "with Self" trick as @Anders Melander pointed out earlier. It is worth noting, that, unfortunately, these functions cannot be implemented efficiently at the TStrings level.

 

The underlying data structure of SynEdit is a TStrings descendent.  Whether that is a good choice is debatable, but It does define such functions (InsertStrings, DeleteLines) and implements efficiently with a single Move, so you could have a look at SynEdit/Source/SynEditTextBuffer.pas at master · pyscripter/SynEdit to get ideas about how to implement them for TStringList.

 

It is worth creating an enhancement request to the Delphi issue tracker, asking for such functions to be added.  Insert/DeleteStrings could be declared as virtual in TStrings with inefficient implementations and then implemented efficiently in descendent classes. Also, the following procedure

procedure AddStrings(const Strings: array of string); overload;

should be declared virtual and implemented efficient in descendent classes.

Edited Monday at 11:59 PM by pyscripter

[Stefan Glienke 2150]

7 hours ago, pyscripter said:

This is not quite true. TStrings does not define Insert/DeleteStrings but it has an AddStrings function.   This function is inefficient (adds the strings one-by-one), but the operation is enclosed in Begin/EndUpdate, so you only get change notifications before and after the whole operation.  In an analogous way, an InsertStrings function would/should only notify before and after the whole operation.   There is no need/requirement for calling Changing/Changed per item.

Fair enough - however, either only TStringList would have these methods, or they would need to be virtual in TStrings with an unoptimized loop implementation in any inheriting class that desires. FWIW, one of the AddStrings overloads is virtual, but TStringList still does a loop calling InsertItem. Why does it do that? Because InsertItem is also virtual, and any inheriting class can do whatever it wants in there.

 

Providing any non-looping operations would possibly circumvent any custom logic that sits in such an override.