Willicious

From what I can tell, none of the loading is done from the level select treeview. It's all done in LemLevel and LemNeoLevelPack and then the parsed data is merely accessed from LevelSelect. But, if that's the case, then what's causing the slow response times? It's very difficult to narrow it down to any one particular thing without guidance, hence this thread. EDIT: It's possible that, by accessing the data from LevelSelect, that causes it to be parsed in LemLevel. I'm not sure which way around it is.

Yeah, it's not always the best resource for programming, but it can sometimes help with quick solutions (e.g. syntax errors) and can actually help with what you're suggesting (i.e. breaking a block of code down to analyze what it does). It absolutely still requires basic knowledge to be able to make good use of its output though. I'm not a complete beginner by now, and I do understand way more about how things work than I did this time last year (for example, I know the difference between functions, methods, variables, constants, types, classes, and what these are all for). I made my first enum type a couple of weeks ago, for example, and it was satisfying to get it right and have it optimise the code I was working on. That was done with the guidance of another programmer as well. I've been very lucky to have lots of help, and I hope I can one day provide the same for someone else. I generally do take this approach. It's more difficult with some things than others. For range check errors, from what I understand it's if something falls outside the specified bounds, for example: (i = 1 to 10) If I then somehow = 11, that would cause a range check error...?

OK, thanks. It's good to have that confirmed. I'll look at ways to cache the data for later quick loading. The data that can be changed whilst the same instance of the Level Select dialog is open is level records and talisman completion data, which is all user-specific and parsed from a single text file, so this shouldn't be too problematic to re-load on the fly, as long as the actual baseline pack/level info itself is pre-loaded. When you say "optimisations in place that would prevent this from working", what do you mean by "this"? Fair question. The answer is simply because it will likely take me many months to bring the alternative UI into some semblance of usability. I'll need to learn how to load fonts and display them at the correct size for the size of the Window (or load the letters as bitmaps, and then have to increase the size of the base window so that the font can be more hi-res). Along with many other challenges, of course. So, I'd like to optiise the existing Level Select dialog in the meantime. That way, it's better sooner, and if the challenges of a brand new screen prove to be too large, I'll at least know more about how the existing one works and what I can do to optimise it. Also, I'd like to know how to cache info anyway; it may come in useful for other program elements. I'll add an option to the config menu to clear program cache, and make sure to clear it when a new copy is installed. I need to learn how to do it, though.

A Google search for "integer overflow delphi" brought up the answer of using data types that are large enough to contain all values that may possibly be computed and stored in them, and a few responses suggested Int64. ChatGPT seemed to confirm this by suggesting that I change all integer types in the FadeOut procedure to Int64 instead. I went ahead and made this change, and still got the integer overflow error. So, I then applied the fix suggested by Kas Ob. (this one:) if GetTickCount > EndTickCount then // prevent integer overflow Break; And, this fixed the error. So, congratulation Kas Ob! What this has done is confirm to me that asking a person, and giving my specific case, is vastly more preferable to generic Googling and even using AI tools. How can I possibly have known that this would fix the issue? The best developer tools on the planet didn't even help solve the problem. In the end, it took a more experienced programmer applying their existing knowledge to the specific issue I was encountering. As a hobbyist with limited time to spend programming, how can I acquire that knowledge? We now have a range check error. Again, I tried googling "range check error delphi" and asking ChatGPT for a solution, neither has produced anything which fixes the issue: procedure TRenderer.ApplyRemovedTerrain(X, Y, W, H: Integer); var PhysicsArrPtr, TerrLayerArrPtr: PColor32Array; cx, cy: Integer; MapWidth: Integer; // Width of the total PhysicsMap begin // This has two applications: // - Removing all non-solid pixels from rlTerrain (possibly created by blending) // - Removed pixels from PhysicsMap copied to rlTerrain (called when applying a mask from LemGame via RenderInterface) PhysicsArrPtr := PhysicsMap.Bits; TerrLayerArrPtr := fLayers[rlTerrain].Bits; MapWidth := PhysicsMap.Width; for cy := Y to (Y+H-1) do begin if cy < 0 then Continue; if cy >= PhysicsMap.Height then Break; for cx := X to (X+W-1) do begin if cx < 0 then Continue; if cx >= MapWidth then Break; if PhysicsArrPtr[cy * MapWidth + cx] and PM_SOLID = 0 then begin if GameParams.HighResolution then begin TerrLayerArrPtr[(cy * MapWidth * 4) + (cx * 2)] := 0; TerrLayerArrPtr[(cy * MapWidth * 4) + (cx * 2) + 1] := 0; <---------------------- this line here TerrLayerArrPtr[((cy * 2) + 1) * (MapWidth * 2) + (cx * 2)] := 0; TerrLayerArrPtr[((cy * 2) + 1) * (MapWidth * 2) + (cx * 2) + 1] := 0; end else TerrLayerArrPtr[cy * MapWidth + cx] := 0; end; end; end; end; My question here would be: what exactly should I be Googling for, and how will I know if what I've found is helpful? If the answer is: "by testing out every single possible thing I find from a generic Google search", that's not exactly encouraging. Stack Overflow, in particular, is full of unspecific answers which usually require an existing level of knowledge and experience to understand in the first place, let alone put into practice for testing. Don't get me wrong, I've been using Google searches, this Forum and various AI tools over the past year to get to where I am now with SuperLemmix, so I can absolutely use these resources effectively, up to a point. When it comes to not actually knowing what I'm supposed to be searching for, or how to know whether the answer is at least relevant if not also correct, this is where I probably need a bit more specific help/mentoring. I've considered paying for lessons many times, because ultimately these are the biggest barriers to progress.

Yes, let's get the thread back on track. The main thing I need help with here is fixing the slow-responding level select menu & treeview - the fixes that have been applied so far have helped for sure, but haven't eliminated the slow response times. I'm considering adding a progress bar, but if I do this then I'd want to load all information to the treeview at the start, rather than adding some when nodes are clicked. Node clicking should, then, access pre-loaded info (if this is at all possible). Please bear in mind, those that have offered suggestions so far, that I need step-by-step instructions if possible. If you say something like "just use a 3rd party treeview", that means nothing to me without instructions as to where to get it and how to intregrate it into the project. I understand that not everyone has time to type out instructions to a newbie though, so... it's fine if you don't, but chances are I probably won't get around to implementing your suggestion. Regarding general optimization though, it'd be nice to get some help with what to do with Overflow & Range checks. At present, they show a popup and point to a code area (this is great!), but I don't really know what to do from there.

Thanks for the fix, I applied it and it fixed both of the leaks! I likely wouldn't have spotted this either, because on the surface it did originally appear to be the case that the cursors were being freed correctly. I also enabled Overflow and Range checking; so far, this Integer overflow is reported: procedure TGameBaseScreen.FadeOut; var RemainingTime: integer; OldRemainingTime: integer; EndTickCount: Cardinal; const MAX_TIME = 320; // mS begin EndTickCount := GetTickCount + MAX_TIME; OldRemainingTime := 0; RemainingTime := MAX_TIME; ScreenImg.Bitmap.DrawMode := dmBlend; // So MasterAlpha is used to draw the bitmap while (RemainingTime >= 0) do begin if (RemainingTime <> OldRemainingTime) then begin ScreenImg.Bitmap.MasterAlpha := MulDiv(255, RemainingTime, MAX_TIME); ScreenImg.Update; OldRemainingTime := RemainingTime; end else Sleep(1); RemainingTime := EndTickCount - GetTickCount; <--------------------------------------- this line here end; Application.ProcessMessages; end;

Here are the screenshots as promised. This is graph & threads on opening the app: And then again a few mins later after loading and playing a level: 2 more memory leaks after playing, these I'll try to fix using the same method as msohn: Ignore this one, added it by accident and for some reason the site won't let me delete it:

That's Lemmix, the original program. There has been NeoLemmix and now SuperLemmix since then. You can get the SuperLemmix source code here (it builds in RAD 10.4, not sure about later versions): https://github.com/Willicious/SuperLemmixPlayer I'll take another look at PE later today, thanks again for your help and advice.

Whenever people say anything like this, I have to think: when will it not be too advanced for me? It's either advanced or it isn't. I'm either up to the challenge or I'm not. I'll only know if I go ahead and try it. With that said, allow me to be the first to admit that I am definitely no more than a novice programmer at best, my interest and experience in it is all from a hobbyist point of view. But, take a look at what I've achieved with SuperLemmix over the past year. You might be surprised at what a beginner is capable of given enough determination and some strong goals!

As an aside, I'm looking at possibly implementing a totally new level select screen which is based on the same look and feel of the existing menu screens. Here's a mockup. This is dreamland at the moment, I only have vague ideas of how I might go about this (and I'd still need to solve the problem of parsing level info): (The row of text "buttons" along the bottom would change according to whether a pack or a level is selected - it wouldn't display both rows at the same time). Also, please excuse the haphazard layout, it's really a basic mockup at the moment.

OK, so a TTimer would be better than using GetTickCount for running the animation (and, of course, handle it in a separate procedure)? 💯 Totally agree with this. I've recently implemented visual feedback for several features in both the SLX Player and Editor (status bar, panel hints/messages and progress bars to name a few), so that the user knows that something is happening. With loading levels though, it's probably too quick to warrant a progress bar but slow enough to be noticeable.

I'm not sure tbh. The project is many programmers' work, I am the most recent. I have the least experience and am working solo. I wouldn't know exactly what steps have been taken to ensure that the message queue isn't overloaded unnecessarily. It already does this. The user data is saved to a text file and then loaded into the treeview, so it always opens at the last-played (or next unsolved) level (depending on user options). Before I do this, just wanted to make sure that I'll deliberately do the correct thing next time rather than indeliberately. How can I make sure I'm showing the main thread? The menu screens play short sound effects when buttons are clicked. All sounds are handled through BASS, so... from what you're saying, this shouldn't be causing any issues (if I've understood correctly - i.e. the UI unresponsiveness is due to synchronous rather than asynchronous {i.e. BASS} operations)? It's possible that the main thread is doing more than handling UI. I'll get those screenshots to you once I know what I'm doing. Got the pull request, thanks for doing this! As I work on this project solo it's great to get help from more experienced devs. And, it's good to know that any memory leaks I see from now on will be because of something I've done (and so can quickly fix myself). OK... so, perhaps I can implement the scroller animation so it isn't handled by ApplicationIdle. Which would be better for time logic, TTimer or GetTickCount? Meanwhile, could caching all level info help? Rather than always parsing the same level info over and over every time Level Select is loaded, SLX could instead load this data from cache and then check for any new additions (something like that, maybe?). The userdata file (the one which stores save data, records, talismans, etc) would still need to be parsed as that will necessarily change each time SLX is run and played.

Thanks for this! Here are the screenshots: The jump up is when I click the Level Select button in the Main Menu. It currently takes about 2 seconds to open (it also takes about 2 seconds to close as well): This is the Threads tab. Not sure exactly what any of this means, but I'm willing to learn if it will help speed up the program and optimise memory usage: I have the Debug build output a dialog with memory leaks info. I did some stuff in the Level Select screen before closing the program. Here is the output dialog showing memory leaks. I'm not sure exactly how to address these, because I'm not sure what they're pointing to. Is there a way I can get it to output more specific information that points to code areas so I can fix them?

When you say "pre-made", does this mean the items have to be pre-loaded, or can this be a blank tree which is populated when loading the program? If the former, then doesn't that mean a lot of manual text editing work for the user? If the latter, how does it differ from/improve on the current stock treeview?

Great! I see what you've done. Thanks for this, much appreciated 🎉

I'm using SourceTree. I found the "Apply Patch" option, but when I load the .patch file SourceTree crashes before I can apply it. Not to worry though, I applied the changes manually and it does seem a bit snappier now. The Expanded event handler is a much better place to call SetInfo, for sure. The only issue is that using fLastGroup to store the Talisman info doesn't seem to work. If you click the pack (i.e. to display the Talisman list), then click a level within that pack, then click the pack again, the Talisman list doesn't load a second time. I get what you were trying though; it's things like this which make the program so much more cumbersome than it appears at first glance. At some point I plan to have a really good in-depth look at the Level Select in general. I'm thinking of building a custom screen for it using the existing Menu screen logic.

Thanks for this! This is the first time anyone has sent me a .patch, how do I apply it?

It displays the information for the currently-selected tree node within the level select menu's display area. But, this isn't the issue; the issue is that there is a delay between clicking and SetInfo being called. So, whatever SetInfo does, it can't possibly be responsible for the delay. This sounds like it might be helpful for keeping the treeview snappy, but does it cause slower loading times initially? To be sure I understand: do you mean pause in-game, or pause the menu? I'm not sure how to pause the menu, but the game has a "P" hotkey for pause, which is handy for debugging game stuff. The menu is much harder to debug since it can't be "paused", as such. Maybe you mean something else though, in which case this sounds like it might be excellent advice! 👍 The call stack screenshot in this reply was always with ApplicationIdle commented out; we were trying to eliminate it at that point. From what I've seen, sleep is only ever used for between-screen stuff and to relax the CPU (and, in this ApplicationIdle procedure, the only function of which seems to be to update the scroller animation). If I comment out this particular Sleep(1) call, it doesn't seem to make any difference to anything, so maybe it's OK to remove it anyway! When you say "Main thread", it's difficult to know exacly what the main thread is in this project. There are > 46000 lines of code across about 40 separate units! Your advice about not using ApplicationIdle too much seems good, though. Maybe the scroller could be its own procedure. We still need to update it, though. Do you suggest using a TTimer instead for this? Thanks for taking the time to build and test, appeciate it Yes, the SetInfo call is probably a bit much for the click handlers. The reason I did this is because the node labels (i.e. the names of the levels) don't load when opening a new "tree" - so, for example, clicking on "SuperLemmix Welcome Pack" would open all the nodes for that pack, but the level titles would all be blank until a node was manually selected. I realise that calling SetInfo is overkill for this, but I tried and couldn't find any other way to do it. And yes, definitely, I'd like to cache stuff so SetInfo does the bare minimum. This is a good shout. Not sure where to start with it, though. I will sit down with it at some point and see what's possible.

I did try this actually (i.e. explicitly disabling idle and then re-enabling Idle before and after the Level Select modal call), but the problem still persists. Can we move away from thinking it has anything to do with ApplicationIdle? That theory has been disproven simply by commenting out the entire procedure. I realise that the call stack doesn't really point to anything else, but maybe the problem isn't actually with the program. It could be a Windows thing, a VLC treeview thing, etc...?

The image that's being clicked is to show the modal (Level Select) in the first place, which is where the treeview is displayed. The call stack doesn't show the treeview click event because I've inserted a breakpoint at the treeview click even handler call, and I'm pausing the program between clicking the treeview and the breakpoint happening - so, we never see what the treeview is calling in the call stack. That's OK, though; the point of this exercise is to determine what's causing the delay between treeview click and treeview click event handling. Regarding the ApplicationIdle procedure, I find it unlikely that this is the problem. As I stated in my previous post (important parts highlighted): So, hopefully we can agree that the ApplicationIdle procedure is unlikely (although not uncertain) to be the problem. To address your questions, though: I didn't write the code, I'm working on a fork of no fewer than 4 other programmers' work. So, the codebase is full of inconsistencies and calls that seemingly have no purpose, etc. As a relative beginner (I've only been working on it for about a year and a half), it makes the process that bit more difficult, but I do get to learn lots about how not to do things. The ApplicationIdle procedure is intended to do a few initial checks upon first loading the program (is it a new version?, does a user config file exist?), and then handle the situation accordingly. Generally this is only relevant the very first time a brand new version is opened on a user's system. Beyond that, it handles updating the scroller reel (a simple animation on the menu screen). My best guess is that the sleep allows time for the reel to update, but I could be wrong about this. The original developer is difficult to get hold of, and usually can't remember off the top of their head why certain calls have been made, so I have to make (un!)educated guesses and generally leave things as they are unless I have a good reason to change them. I didn't know I had to, simple as that really. Best guess is that some previous version of the unit did set Done to true, but this was later removed (perhaps inadvertently). I'm unsure of exactly what to do about this, would you suggest I simply set remove the call to set it to false, or explicitly set it to true somewhere else (possibly when closing the menu screen)?

Commented out the entire ApplicationIdle procedure (including the call for Sleep(1)). The result is that the scroller animation in the Menu screen doesn't update. Otherwise, the program runs normally. However, the problem (i.e. slow-responding Level Select treeview) persists. Here's the resulting call stack (i.e. between physical mouse click and treeview click event handler): Here's the ApplicationIdle procedure: procedure TGameMenuScreen.ApplicationIdle(Sender: TObject; var Done: Boolean); begin if fCleanInstallFail then begin DisableIdle; fCleanInstallFail := false; ShowMessage('It appears you have installed this version of SuperLemmix over ' + 'an older major version. It is recommended that you perform a ' + 'clean install of SuperLemmix whenever updating to a new major ' + 'version. If you encounter any bugs, especially relating to ' + 'styles, please test with a fresh install before reporting them.'); fLastReelUpdateTickCount := GetTickCount64; EnableIdle; end else if not GameParams.LoadedConfig then begin DisableIdle; GameParams.LoadedConfig := true; ShowSetupMenu; EnableIdle; end else if not fDisableScroller then UpdateReel; Done := false; Sleep(1); end; Any suggestions welcome...

I'll definitely give it another try at some point. For now I'm trying to get a few things stable before I introduce any new project elements (hence trying to get the treeview code optimised).

I see. I'm guessing there's no easy answer to this particular problem. Could I maybe limit the number of messages being sent in some way, or prioritize treeview stuff when the form is active? Managed to grab the Pause button from the Menu just in time. Anyone's guess why there's no hotkey for this; someone at Embarcadero didn't have their Weetabix the day that feature was added, clearly. Anyways, here's a screen shot of what I can see. CPU processing and no call stack history: However, did get this when I stepped over with F8: Very unlikely. It's a 32-bit program which (as far I understand) doesn't support multi-threading. It's pretty ancient by today's standards so unlikely to be trying to pull those sorts of manoeuvres (although, I could be wrong about that). This is more likely. Caching sounds like a very good idea, since each time a treeview item is clicked it has to reload a bunch of info. As far as I can tell, all assets are loaded on form creation, but there could be some re-loading/re-drawing going on somewhere. Thing is though, if the delay is happening before SetInfo is even being called, then it stands to reason that isn't the cause of it. Still, I'd definitely be up for learning how to cache stuff if it's a generally helpful thing to do to save memory. Yes. I've had someone else suggest that I use a better treeview. I'm guessing I'd install it like a plugin, like Bass.dll or GR32 (or madExcept)...? Would definitely be up for exploring that. See above screenshot for debug output on pausing. I'm probably not using the debugger correctly though, because it's gone straight to CPU stuff. Meanwhile, madExcept found nothing, so I've uninstalled. It probably is the message queue, from what's being said. --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Thanks for the help so far everyone, I know I'm a bit of a novice; it's nice to have the suggestions.

Last resort as it takes a while to paste in debug strings to every procedure, but if all else fails this is probably what I'll do In fact, calling it now that this is probably what I'll end up having to do

This one is very useful to know in general, thanks. Does Program Pause have a hotkey? Since I'm needing to debug input, I need to be able to pause pretty much immediately after I've clicked the treeview.