Jump to content
Mike Torrettinni

List of usable RegEx for source code

Recommended Posts

6 hours ago, David Heffernan said:

It's always a good idea to minimise contention on process wide locks, which is why it is best not to call SetLength over and over when that can readily be avoided. 

FastMM already reserves some space after a string/array so manual reservations are unneeded where it is used. F.ex., the following test case

 

procedure TForm1.Button2Click(Sender: TObject);
var arr: array of byte;
i: Integer;
begin
  for i := 0 to 30 do
    SetLength(arr, length(arr)+1);
end;

only caused four reallocations

Share this post


Link to post
36 minutes ago, Fr0sT.Brutal said:

FastMM already reserves some space after a string/array so manual reservations are unneeded where it is used. F.ex., the following test case

 


procedure TForm1.Button2Click(Sender: TObject);
var arr: array of byte;
i: Integer;
begin
  for i := 0 to 30 do
    SetLength(arr, length(arr)+1);
end;

only caused four reallocations

Isn't that going to claim a global lock 31 times?

Share this post


Link to post
10 hours ago, David Heffernan said:

Isn't that going to claim a global lock 31 times?

No - when the already allocated block is large enough it doesn't do much.

And when it has to actually allocate it just does a lock cmpxchg on a bool field of the block types - when its already locked by another thread it simply tries the next bigger one.

Speaking of FastMM4 allocation was never the problem with many threads because of this mechanism - it could only lead to a bit memory overhead because some small blocks might be little overallocated because it had to go past the currently locked sub allocators. The issue was in deallocating because that needs of course to the allocator it the memory belongs to. This is what Pierre addressed in V5.

 

FastMM having *a* global lock is a myth.

Edited by Stefan Glienke
  • Like 1
  • Thanks 3

Share this post


Link to post
On 4/2/2021 at 6:58 PM, David Heffernan said:

Isn't that going to claim a global lock 31 times?

Stefan answered better than I could. In addition, don't you do premature optimization here? In numerous cases reallocs are not an issue even in multithread apps, and really heavy multithread tasks will try avoiding allocs at all.

Share this post


Link to post
2 hours ago, Fr0sT.Brutal said:

don't you do premature optimization here?

David unfortunately suffers from a severe chronic heap allocation intolerance 😞

Joking aside: awareness of (hidden) heap allocations is usually a good thing though.

Share this post


Link to post

The fact that the term "scalable memory manager" exists shows that heap allocation is a particular issue for multi-threaded programs.

 

3 hours ago, Fr0sT.Brutal said:

really heavy multithread tasks will try avoiding allocs at all

 

That's the same point that I have been making.

Share this post


Link to post
On 3/31/2021 at 11:26 AM, Mike Torrettinni said:

OK, that makes sense. I use 10.2.3 so new LSP can't help here. You never used any RegEx on your code?

I never use any RegEx in my code. :classic_smile:

Share this post


Link to post
7 hours ago, Joseph MItzen said:

I never use any RegEx in my code.

As the old saying goes, used regex to solve a problem? now you have two problems! 🤣

 

That said, I use regex extensively both in Delphi (I wrote the original System.RegularExpressions code) and .Net (keep it simple, and it's not for parsing html!) - but using it to find problems with my code? ah nope, for that I use Eyeballs 1.0 and static analysis tools like FixInsight and Pascal Analyzer (yes I know they have their limitations).  Plus I guess I have learned a few things over the years that I apply in new code I write. As for the old code, well refactoring is a work in progress.

 

Yesterday I rewrote a lexer/parser framework I initially wrote 10 years ago due to performance issues,  and that issue was identified through profiling (see the thread on vtune). Switching to records took the memory manager from being a big % to insignificant (ie no longer shows up in the profiler). I thought I might get something like a 10% improvement, but was pleasantly surprised to see a 30% improvement! Those sort of gains don't come often in a mature code base.  

  • Like 2

Share this post


Link to post
Guest

I HATE it when there is an interesting discussion under a non-topic (discord between OP title and what's actually being discussed).

OT is trying to make code-review with a "set of regexps". I would not even glance at that because i have Delphi AST (FLOSS, BTW).

 

Pls create new treads. You do let you code do it a lot 🙂

Share this post


Link to post

It would be nice if IDE would allow to search multiple expressions at once, like regex list:

search regex(s) = "SetLength.*\(.*Length\(.*\).*\+.*1 "," SetLength.*\(.*High\(.*\).*\+.*1 "," 0 to Length\(.*\).do "," 0 to .*Count[ ]do "," raise e[a-z]*\( "

 

So you can run such check every now and then with simple copy paste of a list. File Locator/Agent Ransack is missing such options, too, shame.

Share this post


Link to post

There are many linux tools for windows or there is the linux subsystem for windows, or even powershell if you are masochist, why would you use the very limited IDE search?

Share this post


Link to post
2 hours ago, Attila Kovacs said:

There are many linux tools for windows or there is the linux subsystem for windows, or even powershell if you are masochist, why would you use the very limited IDE search?

Because many of us like to work in the IDE and prefer it to use tools that integrate with it. Of course one could add a commandline grep to the IDE's tools menu to automatically run it on the current unit, but even that would not result in a clickable list of possible offending lines.

Share this post


Link to post

Is Delphi IDE using some different than normal regex?

 

Why would IDE not work with this expression:

 

to find all SetLength(arr, Length(arr) + 1)

 

FileLocator and regexr.com work with this: SetLength\s*\(.*Length\(.*\)\s*\+\s*1

But for Delphi IDE I need to add .* after some \s* : SetLength\s*\(.*Length\(.*\)\s*.*\+\s*.*1

 

not found: SetLength\s*\(.*Length\(.*\)\s*\+\s*1

image.thumb.png.f0643dc510a05b7e14cfd6a4dcee3fef.png

 

works Ok with extra \s*.*: SetLength\s*\(.*Length\(.*\)\s*.*\+\s*.*1

image.png.d81b3f218a5142880b3aa887a68688f5.png

 

 

 

 

Share this post


Link to post
4 hours ago, dummzeuch said:

but even that would not result in a clickable list of possible offending lines

How would you click on a line which can't be evaluated with the built in search? 🙂

 

Share this post


Link to post
10 hours ago, Attila Kovacs said:

How would you click on a line which can't be evaluated with the built in search? 🙂

 

I'd use GExperts Grep, of course.

 

But the question was:

17 hours ago, Attila Kovacs said:

why would you use the very limited IDE search?

I would prefer the IDE search over more powerful external tools if it can somehow still fit the bill for the convenience of easily navigating to the found lines.

Share this post


Link to post
30 minutes ago, dummzeuch said:

easily navigating to the found lines.

Agree for recurring tasks.

I think a listview on a dockform and a code navigaion routine is not that hard.

Share this post


Link to post
56 minutes ago, Attila Kovacs said:

I think a listview on a dockform and a code navigaion routine is not that hard.

No, it's actually pretty easy. Unfortunately I know of no existing way to call an external tool and put its output into such a list view. But it might be worth writing one.

 

Share this post


Link to post

OK, the expressions can become quite hard to manage, so I can see why people are not really that enthusiastic about regex, but here is now my complete list:

 

SetLength\s*\(.*Length\(.*\)\s*\+\s*1  -> SetLength(arr, Length(arr) + 1)
SetLength\s*\(.*High\s*\(.*\)\s*\+\s*1 -> SetLength(arr, High(arr) + 1)
0\s*to\s*.*Count\s*do                  -> 0 to List.Count do
0\s*to\s*.*Length\s*\(.*\)\s*do        -> 0 to Length(arr) do
raise Exception\(                      -> raise Exception() (missing .Create like in raise Exception.Create)
Exception\.Create\(                    -> Exception.Create( (verify that Raise is in front )

Full exp:
(SetLength\s*\(.*Length\(.*\)\s*\+\s*1)|(SetLength\s*\(.*High\s*\(.*\)\s*\+\s*1)|(0\s*to\s*.*Count\s*do)|(0\s*to\s*.*Length\s*\(.*\)\s*do)|(raise Exception\()|(Exception\.Create\()

 

And of course Delphi's IDE is special and requires a little subtle changes:

Delphi IDE:
SetLength\s*\(.*Length\(.*\)\s*.*\+\s*.*1
SetLength\s*\(.*High\s*\(.*\)\s*.*\+\s*.*1
0\s*.*to\s*.*.*Count\s*.*do
0\s*.*to\s*.*Length\s*\(.*\)\s*.*do

Full exp:
(SetLength\s*\(.*Length\(.*\)\s*.*\+\s*.*1)|(SetLength\s*\(.*High\s*\(.*\)\s*.*\+\s*.*1)|(0\s*.*to\s*.*.*Count\s*.*do)|(0\s*.*to\s*.*Length\s*\(.*\)\s*.*do)|(raise Exception\()|(Exception\.Create\()

 

  • Like 1

Share this post


Link to post

I got another one, although this one is probably useful for small number of developers who prefer begin and end in their own lines:

 

((if)|(while)|(for)).*\sbegin

 

This finds all lines where begin is in the same line as if/for/while statements.

 

I prefer:

if condition then
begin
  ...
  ...
end;

 

and not the 'advanced/pro' way:

if condition then begin
  ...
  ...
end;

 

For me, easy readable is preferred over reducing 1 line of code.

Share this post


Link to post

A new one:

trim(left|right|)\(trim(left|right|)\(

this will check for all combinations of TrimLeft(TrimRight())...

 

I did find a few of these in some open source projects,

 

There is one that has it's own LTrim and RTrim implementations, I assume they were implemented before TrimLeft/Right were introduced in Delphi. Of course it's using them as LTrim(RTrim(str)).

 

  • Haha 1

Share this post


Link to post

If you search for Regex:

uppercase\(.*lowercase\(

There was one that used these functions like this:

if UpperCase(LowerCase(s)) = UppperCase(s) then

which I assume has something to do with utf8 checking, but looks very odd.

 

there are plenty of examples in open source projects where this is used to capitalize first characters, like:

Value := UpperCase(Copy(Value, 1, 1)) + LowerCase(Copy(Value, 2, Length(Value)));

 

The reversed

lowercase\(.*uppercase\(

was not found. Pretty good but will still make the list, probably! 🙂

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×