List of usable RegEx for source code

[Fr0sT.Brutal 900]

6 hours ago, David Heffernan said:

It's always a good idea to minimise contention on process wide locks, which is why it is best not to call SetLength over and over when that can readily be avoided. 

FastMM already reserves some space after a string/array so manual reservations are unneeded where it is used. F.ex., the following test case

 

procedure TForm1.Button2Click(Sender: TObject);
var arr: array of byte;
i: Integer;
begin
  for i := 0 to 30 do
    SetLength(arr, length(arr)+1);
end;

only caused four reallocations

[David Heffernan 2345]

36 minutes ago, Fr0sT.Brutal said:

FastMM already reserves some space after a string/array so manual reservations are unneeded where it is used. F.ex., the following test case

 

procedure TForm1.Button2Click(Sender: TObject);
var arr: array of byte;
i: Integer;
begin
  for i := 0 to 30 do
    SetLength(arr, length(arr)+1);
end;

only caused four reallocations

Isn't that going to claim a global lock 31 times?

[Stefan Glienke 2005]

10 hours ago, David Heffernan said:

Isn't that going to claim a global lock 31 times?

No - when the already allocated block is large enough it doesn't do much.

And when it has to actually allocate it just does a lock cmpxchg on a bool field of the block types - when its already locked by another thread it simply tries the next bigger one.

Speaking of FastMM4 allocation was never the problem with many threads because of this mechanism - it could only lead to a bit memory overhead because some small blocks might be little overallocated because it had to go past the currently locked sub allocators. The issue was in deallocating because that needs of course to the allocator it the memory belongs to. This is what Pierre addressed in V5.

 

FastMM having *a* global lock is a myth.

Edited April 3, 2021 by Stefan Glienke

[Fr0sT.Brutal 900]

On 4/2/2021 at 6:58 PM, David Heffernan said:

Isn't that going to claim a global lock 31 times?

Stefan answered better than I could. In addition, don't you do premature optimization here? In numerous cases reallocs are not an issue even in multithread apps, and really heavy multithread tasks will try avoiding allocs at all.

[Stefan Glienke 2005]

2 hours ago, Fr0sT.Brutal said:

don't you do premature optimization here?

David unfortunately suffers from a severe chronic heap allocation intolerance 😞

Joking aside: awareness of (hidden) heap allocations is usually a good thing though.

[David Heffernan 2345]

The fact that the term "scalable memory manager" exists shows that heap allocation is a particular issue for multi-threaded programs.

 

3 hours ago, Fr0sT.Brutal said:

really heavy multithread tasks will try avoiding allocs at all

 

That's the same point that I have been making.

[Joseph MItzen 251]

On 3/31/2021 at 11:26 AM, Mike Torrettinni said:

OK, that makes sense. I use 10.2.3 so new LSP can't help here. You never used any RegEx on your code?

I never use any RegEx in my code.

[Stefan Glienke 2005]

6 hours ago, Joseph MItzen said:

I never use any RegEx in my code.

Not even to parse HTML? 

[Vincent Parrett 750]

7 hours ago, Joseph MItzen said:

I never use any RegEx in my code.

As the old saying goes, used regex to solve a problem? now you have two problems! 🤣

 

That said, I use regex extensively both in Delphi (I wrote the original System.RegularExpressions code) and .Net (keep it simple, and it's not for parsing html!) - but using it to find problems with my code? ah nope, for that I use Eyeballs 1.0 and static analysis tools like FixInsight and Pascal Analyzer (yes I know they have their limitations).  Plus I guess I have learned a few things over the years that I apply in new code I write. As for the old code, well refactoring is a work in progress.

 

Yesterday I rewrote a lexer/parser framework I initially wrote 10 years ago due to performance issues,  and that issue was identified through profiling (see the thread on vtune). Switching to records took the memory manager from being a big % to insignificant (ie no longer shows up in the profiler). I thought I might get something like a 10% improvement, but was pleasantly surprised to see a 30% improvement! Those sort of gains don't come often in a mature code base.  

[Guest]

I HATE it when there is an interesting discussion under a non-topic (discord between OP title and what's actually being discussed).

OT is trying to make code-review with a "set of regexps". I would not even glance at that because i have Delphi AST (FLOSS, BTW).

 

Pls create new treads. You do let you code do it a lot 🙂

[Mike Torrettinni 198]

It would be nice if IDE would allow to search multiple expressions at once, like regex list:

search regex(s) = "SetLength.*\(.*Length\(.*\).*\+.*1 "," SetLength.*\(.*High\(.*\).*\+.*1 "," 0 to Length\(.*\).do "," 0 to .*Count[ ]do "," raise e[a-z]*\( "

 

So you can run such check every now and then with simple copy paste of a list. File Locator/Agent Ransack is missing such options, too, shame.

[0x8000FFFF 22]

You can combine two expressions into one, though.

 

If I were you I'd at least use \s+ instead of spaces and \s*\( instead of .*\(

 

[Attila Kovacs 629]

There are many linux tools for windows or there is the linux subsystem for windows, or even powershell if you are masochist, why would you use the very limited IDE search?

[dummzeuch 1505]

2 hours ago, Attila Kovacs said:

There are many linux tools for windows or there is the linux subsystem for windows, or even powershell if you are masochist, why would you use the very limited IDE search?

Because many of us like to work in the IDE and prefer it to use tools that integrate with it. Of course one could add a commandline grep to the IDE's tools menu to automatically run it on the current unit, but even that would not result in a clickable list of possible offending lines.

[Mike Torrettinni 198]

Is Delphi IDE using some different than normal regex?

 

Why would IDE not work with this expression:

 

to find all SetLength(arr, Length(arr) + 1)

 

FileLocator and regexr.com work with this: SetLength\s*\(.*Length\(.*\)\s*\+\s*1

But for Delphi IDE I need to add .* after some \s* : SetLength\s*\(.*Length\(.*\)\s*.*\+\s*.*1

 

not found: SetLength\s*\(.*Length\(.*\)\s*\+\s*1

 

works Ok with extra \s*.*: SetLength\s*\(.*Length\(.*\)\s*.*\+\s*.*1

 

 

 

 

[Attila Kovacs 629]

4 hours ago, dummzeuch said:

but even that would not result in a clickable list of possible offending lines

How would you click on a line which can't be evaluated with the built in search? 🙂

 

[dummzeuch 1505]

10 hours ago, Attila Kovacs said:

How would you click on a line which can't be evaluated with the built in search? 🙂

 

I'd use GExperts Grep, of course.

 

But the question was:

17 hours ago, Attila Kovacs said:

why would you use the very limited IDE search?

I would prefer the IDE search over more powerful external tools if it can somehow still fit the bill for the convenience of easily navigating to the found lines.

[Attila Kovacs 629]

30 minutes ago, dummzeuch said:

easily navigating to the found lines.

Agree for recurring tasks.

I think a listview on a dockform and a code navigaion routine is not that hard.

[dummzeuch 1505]

56 minutes ago, Attila Kovacs said:

I think a listview on a dockform and a code navigaion routine is not that hard.

No, it's actually pretty easy. Unfortunately I know of no existing way to call an external tool and put its output into such a list view. But it might be worth writing one.

 

[Mike Torrettinni 198]

OK, the expressions can become quite hard to manage, so I can see why people are not really that enthusiastic about regex, but here is now my complete list:

 

SetLength\s*\(.*Length\(.*\)\s*\+\s*1  -> SetLength(arr, Length(arr) + 1)
SetLength\s*\(.*High\s*\(.*\)\s*\+\s*1 -> SetLength(arr, High(arr) + 1)
0\s*to\s*.*Count\s*do                  -> 0 to List.Count do
0\s*to\s*.*Length\s*\(.*\)\s*do        -> 0 to Length(arr) do
raise Exception\(                      -> raise Exception() (missing .Create like in raise Exception.Create)
Exception\.Create\(                    -> Exception.Create( (verify that Raise is in front )

Full exp:
(SetLength\s*\(.*Length\(.*\)\s*\+\s*1)|(SetLength\s*\(.*High\s*\(.*\)\s*\+\s*1)|(0\s*to\s*.*Count\s*do)|(0\s*to\s*.*Length\s*\(.*\)\s*do)|(raise Exception\()|(Exception\.Create\()

 

And of course Delphi's IDE is special and requires a little subtle changes:

Delphi IDE:
SetLength\s*\(.*Length\(.*\)\s*.*\+\s*.*1
SetLength\s*\(.*High\s*\(.*\)\s*.*\+\s*.*1
0\s*.*to\s*.*.*Count\s*.*do
0\s*.*to\s*.*Length\s*\(.*\)\s*.*do

Full exp:
(SetLength\s*\(.*Length\(.*\)\s*.*\+\s*.*1)|(SetLength\s*\(.*High\s*\(.*\)\s*.*\+\s*.*1)|(0\s*.*to\s*.*.*Count\s*.*do)|(0\s*.*to\s*.*Length\s*\(.*\)\s*.*do)|(raise Exception\()|(Exception\.Create\()

 

[Mike Torrettinni 198]

I got another one, although this one is probably useful for small number of developers who prefer begin and end in their own lines:

 

((if)|(while)|(for)).*\sbegin

 

This finds all lines where begin is in the same line as if/for/while statements.

 

I prefer:

if condition then
begin
  ...
  ...
end;

 

and not the 'advanced/pro' way:

if condition then begin
  ...
  ...
end;

 

For me, easy readable is preferred over reducing 1 line of code.

[Mike Torrettinni 198]

A new one:

trim(left|right|)\(trim(left|right|)\(

this will check for all combinations of TrimLeft(TrimRight())...

 

I did find a few of these in some open source projects,

 

There is one that has it's own LTrim and RTrim implementations, I assume they were implemented before TrimLeft/Right were introduced in Delphi. Of course it's using them as LTrim(RTrim(str)).

 

[Mike Torrettinni 198]

If you search for Regex:

uppercase\(.*lowercase\(

There was one that used these functions like this:

if UpperCase(LowerCase(s)) = UppperCase(s) then

which I assume has something to do with utf8 checking, but looks very odd.

 

there are plenty of examples in open source projects where this is used to capitalize first characters, like:

Value := UpperCase(Copy(Value, 1, 1)) + LowerCase(Copy(Value, 2, Length(Value)));

 

The reversed

lowercase\(.*uppercase\(

was not found. Pretty good but will still make the list, probably! 🙂