how to classify and match error message patterns?

[David Schwartz 415]

Our system does not capture outer-most exceptions, and throws these annoying error messages that are rather hard to decipher on their own.

 

We've been building up a catalog of them, and they fit a collection of patterns. 

 

We'd like to have a form with a field where someone can paste in an error or exception message that shows up in an email, click a button, and it would bring up anything in the database that matches this message pattern and any associated comments.

 

The first cut matches a Format pattern based on a collection of responses rather than a collection of these from the source code. (maybe we could go through and manually reconstruct RegEx patterns for these as there are only around 50 of them.)

 

So, for example, there may be this somewhere in a source file:

 

Raise Exception.Create( Format( '%s - Exception %s - something happened in xyz; %s - %d', [....] ) );

 

If the message in question looks like it fits this "template", then narrow down the search field to those we have documented that also match this template.

 

The second cut looks at specific data values -- so, for example, the 'xyz' and the word following it provided by "%s" or the following "%d"

 

'xyz' might be a table name or process, and the following field identifies a column name and value that it choked on. 

 

In this case, the 'xyz' is the discriminant, while the next to parameters are just whatever was there that it choked on that time.

 

Eg, 'xyz' = ADDRESS TABLE and the following might be "ZIP_CODE = '' is not a valid number"

 

the underlying problem here is that we've got a bunch of StrToInt(...) calls and the developers chose them over StrToIntDef(...) because there may not be a realistic default value to supply. Usually these errors indicate one of two things: (1) a required field in the data file was left empty; or (2) there's an extra or missing comma (field-delim), or sometimes an extra or missing quote. (This is typically referring to a CSV file being imported.)

 

We want something that will search the list of exemplars (that require as little pre-processing as possible) and return any notes and explanations associated with any matches it finds.

 

I was thinking of perhaps adapting a chat bot for this purpose.

 

Does anybody have any experience building something like this that could offer any insights?

 

 

Edited October 13, 2020 by David Schwartz

[Fr0sT.Brutal 897]

With some efforts made to handle Firebird error messages I realized one thing - if you need to automatically handle errors (group, summarize etc) then better not try to extract necessary data from the constructed message, just return this data as-is in an array together with integer error code. HTH

 

Side note: I got rid of ` Raise Exception.Create( Format( ...` in favor of ` Raise Exception.CreateFmt( `* because it's a bit shorter and - what's most important - doesn't add implicit finalization block to a subroutine which significantly slow down execution of subroutines without explicitly used variables of managed types (strings, dynarrays, interfaces).

* moreover, I made trivial overloaded functions that create and return exception objects so I just type `raise Err('error: %s', [ErrMsg])`

Edited October 13, 2020 by Fr0sT.Brutal

[David Schwartz 415]

It sounds like you're suggesting we go back into the code, track down every place it generates these errors, and change the code.

 

That is not an option. We get what we get -- error messages sent to a logfile and put into a note that's sent via email to a group list.

 

And what we get is a bunch of error messages that look like this:

 

Error in ChildDone, C:\Loader\bdm001_44117.6013302546\bdm001_checks_imp.exe: Import error - CUSTOM CODE ERROR: (PostDocumentExtract) Requested image not found - 261101.png

 

We've collected nearly 150 exemplars like this, about half of which have detailed explanations attached to them so far. They're just damned hard to sift through visually, and some are very similar yet distinct. If it was easier to map a given error to an exemplar, our list would be far more useful to people.

 

I want to be able to paste messages we get like these into an edit box and have them parsed sufficiently to match a pattern, then search through the collection of messages and find any that match the pattern and have additional details added below.

 

There's stuff here that's relevant literally (eg, "bdm001_checks_imp.exe"), some that's relevant in a meta-data sense (eg., the 261101.png is "<number>.png") and some that's totally irrelevant (eg., "C:\Loader\bdm001_44117....\" up to the next bdm001_).

 

I'm looking for insights on how to do that.

[Fr0sT.Brutal 897]

Then just use regexp

Error in (.+?), (.+?): Import error - CUSTOM CODE ERROR: (.+?)

to extract message contents. But don't forget to check that substituted values doesn't contain separator values or you can get unexpected results

[Anders Melander 1654]

7 hours ago, David Schwartz said:

I'm looking for insights on how to do that.

It sounds more like you're rubber ducking.

[David Schwartz 415]

15 hours ago, Anders Melander said:

It sounds more like you're rubber ducking.

not sure what that is exactly, but I certainly didn't need to post here to learn the most obvious brute-force approach to solving this.

 

Just to be clear, I know how to go through a list of exemplars and write a regex expression to recognize each one. I'm looking for another solution. If anything, I'd write something to parse the existing exemplars and construct regex expressions from them. I do not like brute-force approaches to problems that require things to be manually updated every time something changes. If I did, I wouldn't waste my time with object-oriented programming; I'd probably stick with assembly language.

 

I don't think anybody noticed my mention of adapting a chat bot for this. Chat bots are designed for parsing similar stuff, from what I can tell. I was thinking maybe someone here has some experience with them and could comment.

 

I'd also like to hear from folks who have experience with writing parsers and might know of similar problems to look at.

Edited October 15, 2020 by David Schwartz