Jump to content

Gary Mugford

Members
  • Content Count

    16
  • Joined

  • Last visited

Community Reputation

0 Neutral

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. Gary Mugford

    Blast from the past: BDE and Win10 W/S

    Before I start, I have to confess to be trying to rewrite the systems software using NexusDB. Progressing, but but not fast enough. In the meantime, our decades-old app is encountering pervasive corruption issues with BDE using Paradox file formats, Windows Server 2013 and NOW mostly workstations running Windows 10 and kept up to date. The problem started with the Fall 2017 update to Windows. Ever since, we have had a regular series of Blob is Modified and Index out of Date errors. We cope by having all seats (approx. 50) get out of the app and associated programs. We run reBuilder which is a modified version of PdxRbd. Then it's back in and we wait for minutes, sometimes an hour when the cycle repeats itself. Sometimes we get people jumping back in early during the rebuild and THAT EATS THE DATABASE with a not that there has been a sharing violation. Luckily, we do zip up a copy of the raw data before hand and can recover in that circumstance. But it's a LITTLE stressful to see major databases disappear from one minute to the next. And when it happens when I'm not there, the place ACTUALLY STOPS WORKING with the computer. Everything goes manual and that is NOT a good thing in today's world. I have theorized that this issue is related to continuing problems with BDE locks and Windows locks going out of sync, if they were ever IN sync. That said, I'm no Windows tech expert. Might be something else. Probably is something else. What I am looking forward is a settings that can make BDE work in the Windows 10 environment the way it STILL WORKS for Windows 7. I don't have the ability to retrograde the OSs for the Work Station, although I spent a night writing up a proposal to do exactly that. The answer was no. Find another way. And I haven't. The non-answer has been to make the workers' endure the stop and start operation that occasionally results in unrecoverable data loss (mostly lines out of history memo fields). And it's getting a LOT worse and I'm still too far away with the NexusDB operation to look my fellow workers in the face. So, I'm reaching out to you DATABASE EXPERTS (give yourself a pat on the back just for coming here to help out people like me) and see if any of you can remember back to when you ran BDE. Or might STILL be running BDE in a small controlled environment. And can suggest a setting, a tool, a whatever. I HAVE started running a virtual box test, but the Workstations all have minimal RAM and having half of it taken up by a vb that is running a short-memory Win7 environment to run the app, turns out to be glacially slow. Although it DOES work. So far, we have two computers set up that way and the hardware guy says he can average three more a week. Which means I would have possible solutions up and running some time in the late spring. At which point, I would hope my direct solution might be ready. So, as much hope as I had originally in the vbox concept, it's NOT the solution for the immediate crisis. Regardless of whether you have an answer, I thank you for taking the time to read this. I hope you and yours have the Merriest of holiday seasons and a sane and safe Happy New Year! GM
  2. Gary Mugford

    Illegal characters causing problems and can't be found

    In the FormCreate procedure, I fill the IllChar array with all the control characters from zero to 31. It is in the line for Kounter := low(IllChar) to high(IllChar) do IllChar[Kounter] := chr(Kounter); In the global vars, I had declared IllChar as an array [0..31] of string, so I THINK I got them all. Then in the compare area, I loop through the array and check with the POS() function. Now, it's eminently possible that in my stuffing line, what I assume is completely filling it fails for a particular code. I haven't EVER used this concept before, since I've always depended on Woll2Woll's masks to keep stuff out that I want out. And I never checked after the fact. And I truly wonder if these data corruptions are causing the insertion of the characters after the fact. In fact, it's the ONLY explanation I have. They are happening. I've seen the EurekaLog errors and I've seen the Rebuilder logs with the notations. But they are like ghosts. When I GET a chance to go looking for them. They're gone. That said, you're suggestion of parsing each PN character by character and doing Char(ThePN) < #32 is worth trying. Approaching any problem that doesn't produce results from a different direction is always worth trying. Thanks for the suggestion. I'll report back here with the results.
  3. Gary Mugford

    Illegal characters causing problems and can't be found

    Sue, Thanks for your response. I've been a fieldByName guy forever. So, I've never thought much about how it retrieves (and potentially changes) the string from the A15 field in the Paradox database. (And yes PN is a constant equal to 'PartNum.' I DO know that the stated problem child has an unusual PartNumber. I remember eons ago putting in a mask for just alphanumerics and a dash. But that changed 'bout five years ago when a client more or less demanded this + part number scheme because they were building out a temporary set of items to fulfill some legal judgement against them ... not having something or other to meet some local code of some sort. It was meant to go away. But they decided after the fact NOT to let it go away. Ergo, a change to the mask to allow the plus sign ended up being permanent. I checked that. So, I'm more or less, of the opinion that it's something evil sneaking into the data. I DID have a problem with one place where the minions were managing to get carriage return characters into a list, rather than just filling in the blank with one long comma-separated list where that was what should have been entering. The miracles of copy and paste) So I had to add a Strip(DeptNums,crLF) into my posting check. I really only found it exporting to XLS where some deptNums were double height in the spreadsheet. Otherwise it was colourless and odourless. The other place where evidence rears it's head is in Rebuilder, which is a customized version of RK Solutions' Pdxrbld. Here's a slightly redacted example for one table in a recent rebuild: Table EXAMPLE - #records: 90802 Index EXAMPLE.PX - Index version does not match table version Index EXAMPLE.XG0 - Secondary Index onF1 is out of date Index EXAMPLE.XG1 - Secondary Index onF2 is out of date Index EXAMPLE.XG2 - Secondary Index onF3 is out of date Index EXAMPLE.XG3 - Secondary Index onF4 is out of date Table EXAMPLE.DB - Alpha Field SVException in record 68422 contains low ascii chars Table EXAMPLE.DB - Alpha Field SVException in record 69147 contains low ascii chars Table EXAMPLE.DB - Alpha Field SVException in record 70147 contains low ascii chars Table EXAMPLE.DB - Forward block pointer in block 2893 is out of range Table EXAMPLE - Errors fixed (#records: 90803) Table EXAMPLE.DB - successfuly packed As you can see, there are three records that trigger a low ascii chars warning. However, by the time I'm alerted to this, the rebuild has changed the database and I'm stuck for a reproducible error to work against. I DO have our Rebuilder zip up the raw data, but it goes to a dated name that gets overwritten all day if there are multiple rebuilds. EVERY TIME I ask for them to look before doing it again, I get blank looks. We hire the cheapest, not the best. (Including, probably, the programmer). When I saw this example, I did change my code to test for all string fields in each record. Came up clean as a toothpick in ready-to-eat cake. A bit frustrating, but c'est la vie. I have to get this all transferred into NexusDB and this problem goes away. Maybe I shouldn't be wasting my time and everybody else's time when it's an issue that sort of fixes itself when something else breaks and Rebuilder gets run. Sure is curious though. Thanks for your DOCUMENTATION code you sent me. I didn't QUITE get it all straight, but I figured out an alternative once I figured out I could get scripts out of the SERVER's SQL context menu where as I had been looking into the SQL found in MANAGER. It was quite the DUH moment. But taking a bit of your code, that creation code and my own stuff, I have something that equates to my needed documentor from BDE. Kicked it all over to the Atomic Scribbler novel writing software and made it into a Documentation Writer too. So, thanks again for help here AND in the past.
  4. Gary Mugford

    "vertical first" vs. "horizontal first"

    It might be relevant to remember the authors of the wizard are Chinese who come from a different reading environment than western European. They read top to bottom, right to left. So, like Uwe says, they would think of row-oriented reading as vertical since you read all of row 1, then all of row 2, etc. And that's vertical from their perspective.
  5. Same problem, same result after David's suggestion. Appreciate the intel. Thank you both for raising the subject and getting a solution.
  6. Down the rabbit hatch again. I'm hoping the smart minds here will continue to try and help me. I am battling to complete a conversion of a BDE project, but it still has to run in the meantime. Written in D7. Been going for close to 30 years dating back to the venerable DOS Paradox 1.11i. Since Microsoft's Fall 2017 Update to Windows 10, our data integrity has been under daily attack. Running a rebuilder session once a day is a good day, seven times in a day is ... all too often. MOSTLY, the problems have been Blob Modified and Index out of Date errors. In both cases, I believe the timing between Win10, SOME Win7 workstations, BDE and the server are just off. Guessing the locking protocols are broken badly. But we persevere. Today, a NEW problem. ONE user, running the app in a Win7 VirtualBox on Win10 (one of my workarounds to try and minimize the BDE issues), got an error that clearly says: "Invalid characters. Field: PartNum" I wondered if it might be a picture mask issue, but no, this was an existing part and once into the system, part numbers are inviolate and can't be edited at all. So, I presume our persistent integrity issues were at play and somehow a low-ascii character had been inserted/appended during a rebuild session. Indeed, in rebuilding logs, I have seen a reference to up to four records needing repairing for low-ascii. So I built a small utility to check for these characters. And found nothing. The error message, very reproducible, and my blank results are ... not mutually compatible. So, I must be doing something wrong in my search for illusive control characters. The array is 0 to 31. Here's the code: procedure TFrmMain.FormCreate(Sender: TObject); begin t.databaseName := 'GM'; t.TableName := 'PARTNUMBERS.DB'; t.active := true; TotalRex := t.RecordCount; for Kounter := low(IllChar) to high(IllChar) do IllChar[Kounter] := chr(Kounter); end; procedure TFrmMain.BtnStartClick(Sender: TObject); var mNote: string; begin CurrRex := 0; with T do begin // disableControls; first; while not eof do begin inc(CurrRex); g.Position := Trunc(100*CurrRex/TotalRex); Application.ProcessMessages; ThePN := fieldByName(PN).asString; mNote := ''; for Kounter := low(IllChar) to high(IllChar) do begin if pos(IllChar[Kounter],ThePN) > 0 then begin if mNote = '' then mNote := PN + ' ... '; mNote := mNote + ' ' + IntToStr(Kounter); end; end; if mNote <> '' then m.Lines.Add(mNote); next; end; // enableControls; end; ShowMessage('Finished'); end; Is it possible that the D7 POS command won't find control characters in strings? Or a sub-set of the control characters that includes the ACTUAL problem characters? Did I goof up the memo lines adding note creation? I slowed the thing down by using a gauge and commenting out the disableControls/enableControls. Ideas appreciated. Thanks in advance, GM
  7. Gary Mugford

    Releasing memory devoted to arrays

    Oh, and bonus, after using the explaining vars to ... slow down the app??? ... compiling didn't throw one of the F2039 errors during about 10 more compiles. Any changes I made before compiling were mostly a LOT less than 2 minutes apart. So, wins all round. Here's the code I used to either slow things down or actually cure the issue: P1Value := fieldByName('Parm1').AsFloat; P2Val := fieldByName('Parm2').AsFloat; // Assert check to see if each value is actually within range ... Assert((P1Value >= 0.00002) and (P1Value <= 20), 'Bad P1Value: ' + fieldByName('Parm1').AsString); Assert((P2Val >= 200) and (P2Val <= 500), 'Bad P2Val: ' + fieldByName('Parm2').AsString); VPP := P1Value * P2Val / 3; When I look at it, it makes no sense to me that this fixes the AV. I admit, repeatedly calling the AsFloat function for the same field contents in a single line is not good. But the assertion NEVER triggered (as I expected) and changing the VPP assignment to use actual numbers rather than functions works, although I don't know why. It worked the original way to PRODUCE the data. But it triggered that exiting AV. I guess we call it a win for Assertions and refactoring, however small. Thanks Kryvich and the rest of you. GM
  8. Gary Mugford

    Releasing memory devoted to arrays

    I'd already ditched the assignment to a default value of zero. It's an old habit of mine to set some default for a variable before starting some formula work. So adding the assertions didn't change things, other than show no records failed the assertion. BUT adding explaining vars to turn the arrays with the lookup function to determine index DID. I've subsequently ran every filter, data range, data sort and flag for showing zero value records that there is. All sequentially within the same run. NO AV. THANK YOU ALL. Your patience has been rewarded. I have my app, and a meeting with the boss in a half-hour for the next task.
  9. Gary Mugford

    Releasing memory devoted to arrays

    Attila, As I've said, I'm hamstrung by employer instructions NOT to post code. As it is, I'm skating on thin ice. I appreciate your frustration. I mirror it. To everybody, There are some new pieces of information. The Range-Check did, in fact, reveal an error that contributed to an AV UPON exit of the program without doing anything. Turns out, one of the minions had entered a Dept name WITH A RETURN attached ... obviously through copying and pasting ... and THAT was being read in the form creation and it was kicking out the AV on ending, sending me on a wild goose chase for the AV there. So, I fixed the code NOT to just over-ride bad departments and keep on going through a Try ... finally block that I never checked to see if it failed because it couldn't ... until it did. I now get an error message if another minion decides creativity can be fun. When I now run the program, I could exit without failure, having fixed the record with the bad dept in it. Happiness? No. I STILL had an AV if I ran the analysis BEFORE I SHUT DOWN. I looked through that button's code for places where things went south. I looked at the assignation of string grid cells looking at a FloatToStr conversion on the run. I commented it out. No fix. I commented out the whole string grid cell assignation code. No fix. I commented out each little part until I came to: VPP := 0; VPP := fieldByName('Parm1').AsFloat * fieldByName('ParmBV').AsFloat / 3; Comment out the second line and everything runs without AV. I changed EACH part to be 1 and EACH time I ran the program and ran the analysis, I got an AV. VPP is a declared Extended variable. The range of values for Parm1 can be in the 0.00002 to 20 area. The second field runs 200-500. Each can go into five decimal values. And the last part is a static integer. The results are maxed out at about 30K. That's well within the scope of Extended, is it not? Even the temporary 100K which is borderline possible is within scope, isn't it? I DO convert the result to a string later JUST with the FloatToStr function. For sticking into a string grid. EVEN with the closing AV, the RESULTS of this line leads to correct results in the string grid. I've manually checked EVERY SINGLE cell. The more I know, the less I understand. And it doesn't help that I have to wait two minutes between compiles or I get the dreaded fatal error F2039 Count not create output file. So, I HAD two AVs although only one would show at Program exit. Now I'm down to one. And it's with basic math. That actually works. Dare I ask for ideas in a thread I've tried to shut down twice? Sure, why not? THanks to everybody for their patience and willingness to help. GM
  10. Gary Mugford

    Releasing memory devoted to arrays

    Kryvich, I did set the Optimization to false and turned on range-checking due to earlier contributions. I will further augment my debugging environment as you have suggest. Thank you. My arrays are one-based for convenience sake. Each array is set high at the count of those depts (or tasks in the other pairing). I have snooped around Lists a bit, going back to Orpheus days. The modern-day implementation seems much more to the point. But as you say, the good old array does get the work done. Identifying a LIST element's index through the string value would be nice, though. GM
  11. Gary Mugford

    Releasing memory devoted to arrays

    Mr. Heffernan, Offers to help should always be welcome and they are here. Debugging this is obviously beyond my skill set at this point. I'm 'Changing things around' hoping to discover where my AV is ACTUALLY percolating, only to flare up at the end of the program. I'm doing it one change at a time, looking for the Eureka moment. I've started in places that look different to me from my decades of using D7. XE7 is a new world. Even the declaration of the one constant array was subject to researching doing so on the internet since I wanted to use variables as the end/high element in the var declarations. I was hoping for the variable arrays, for example, to be arDepts: Arrays [1..DeptCount] of string. It was there, as a side process that I found a different way of declaring the constant array. Interesting and easier to read. BUT, I wasn't going to be able to get the arDepts declared as tightly as I wanted. Ergo, make it 100 and that covers expansion for the rest of my life in the number of departments. It's currently TWO for this project. Eight if it goes company wide. Now, posting function by function here would resolve the curiosity AND the appreciated desire to help that folks here have. It would also be against the explicit instructions I got from my employer NOT to do that. Genericizing the material would suffice to some extent, but again, the Boss said no. He needs me to produce tools in a fast-changing environment that we are currently in. My time is HIS time to do with as he sees fit. Producing a 'simple' case would presume I know generally where things are tripping me up. It's the end of the program. Beyond that ... nothing? Creating a stripped out version of the program would require replacing 3rd party components that I can't publish (with GExperts), eliminating items that would specify my employer and do it all while MAINTAINING where the bug's born. I apologize, but I can't do that. Puttering around with MY spare time seems my best compromise because wasting YOUR time and the time of the others is disrespectful. I hope you understand. Thanks, GM
  12. Gary Mugford

    Releasing memory devoted to arrays

    One more thing ... I left StackOverflow because help seemed to come at a very high price there. I've lurked since. My improvement as a programmer has stagnated because of that. MY FAULT. I should have been less sensitive, better at learning on my own. I found Praxis here and read through the threads, some of which were contentious. But I really appreciated the tone of those dialogs. Help seemed the raison d'etre for the site. And my request for help here has been answered with that same tone of voice. Asking to end the thread seems disrespectful towards the efforts already put forth. Please understand that that is not my purpose. I simply think the process moving forward would waste more of the collective time of the people trying to help. I do not wish to do that, nor insult anybody. THANK YOU for trying to help. I'm simply ill-equipped to aid in the helping of myself. GM
  13. Gary Mugford

    Releasing memory devoted to arrays

    I appreciate each and every suggestion above. The problem with the program is that it's proprietary work and the sum total of the code is several pages. That said, I am looking very much at my use of the arrays. Not surprisingly, each of the two Var arrays are linked, i.e. one being a Dept code, the other the total for that Dept. BUT, I am not NEARLY using the full array. In fact, each run through of the program captures a count of the Depts and I use that to totalize up the Depts. BUT there are only a handful of Depts currently involved. I chose 100 for the limit to protect my backside against successful expansion of this project. I am NOT initializing the arrays with a loop to '' and 0.00 respectively. Don't see a need. I never go PAST the DeptCount, so I'm not accessing unassigned memory. At least I'm PRETTY SURE that's the case. I AM force capturing the INDEX for the particular pairs. See this: function TFrmMain.ReturnDeptIndex(ADept:string): integer; var idx: integer; begin Result := 0; for idx := 1 to DeptCount do begin if ADept = arDepts[idx] then begin Result := idx; break; end; end; end; I assumed there was a direct way to get the index in XE7 but couldn't find any. Thus the kludge above. It's pretty representative of my lack of elegance when it comes to programming. That said, I don't see anywhere where this can induce an error accessing memory it shouldn't. Slow, but successful. Besides, EVEN IF I DO NOT CLICK THE BUTTON that does the analysis, even to just starting the programming and immediately exiting, it throws out the AV. At the top of the function, I include these lines: var tmpIdx: integer; kounter: integer; begin for kounter := 1 to DeptCount do arDeptsTotal[Kounter] := 0.00; ... The function is later called via these lines: tmpIdx := ReturnDeptIndex(TheDept); arDeptsTotal[tmpIdx] := arDeptsTotal[tmpIdx] + LineVal; Again, I'm missing what could be wrong with those lines. I do create a fair number of SQL queries on the fly. There's a bunch of filterable fields, a date range, a results sort picker and a flag check box to include Zero Value lines. It's more "get it to work" code than incredibly featured code. But it runs on for a fair bit. Plus, I couldn't get my boss to give me permission to spend the time and effort to create the stripped down version that all of you are requesting (and reasonably at that). At this very moment, the decision has been to start using the alpha, AV and all. It's an analysis tool rather than a data manipulator. So, the boss is happy to have something that gives him desired info. I'm still changing things around, literally looking at every procedure and function and seeing if I can re-write them and maybe expose the AV but tearing up the hidey-hole. But you can't be expected to do my work for me. So, I am respectfully going to ask that we call a halt to this thread since I'm wasting your time. I appreciate all who contributed. You've give me directions to go on debugging this. And I will explore each suggestion. Thanks, GM
  14. Gary Mugford

    Releasing memory devoted to arrays

    Cristian, I was aware that the code did not represent the full code. Several pages worth. The AV occurs on program exit. From looking at the Error Log, I came to the conclusion my mistake was in not clearing out the memory assigned to arrays. I presented my declaration of the arrays in case I was doing that wrong. As for NOT DOING the things I tried, I ruefully am aware that what I was doing was not working. All that said, it appears something else, other than array finalization is creating the AV. More than possible. As a programmer, I'm self-taught, mostly through examples, it is very possible bordering on probable I've done something careless and that's the source of the AV, Error Log notwithstanding. In my original post, I did point out that the error is frequently the NEXT THING, rather than just what passed, in causing program faults. It does appear that is the case here and I've wasted everybody's time focusing in on the arrays. Thanks for moving me in that direction. GM
  15. Gary Mugford

    Releasing memory devoted to arrays

    Uwe, That had been my supposition in the past. But I get an Access Viol error on program exit and when I look at the stack that trips the error (admittedly a presumption since many errors are actually on the invisible next line, so to speak), I see that the program goes into the system for Halt(), FinalizeUnits, my main form's Finalization, _FinalizeArray and lastly _UStrArrayClr. It trips whether I do anything with the arrays or not. Looking at the assembler version of _UStrArrayClr, there's a MOV line that trips an exception, followed by a DEC and JL instruction and then the ominous warning that the next line is an unaccessible location. Sorry to bring such a newbie question to the forum ... which I've found entertaining, be it at a higher level than I am at. Thanks for helping, GM
×