David Schwartz

I need some code written in either C/C++ or PASCAL (Delphi, Free Pascal, etc.) that suppresses breath noise from divers. I can provide a sample audio file that would be used to demonstrate it works. Ideally, it would be small enough to run on a Raspberry Pi Pico CPU (8MB of storage + 256k of RAM). The "breath noise" is a large burst of mostly white noise produced over audio comms when a diver inhales and/or exhales. (think "Darth Vader's breathing") My thinking is to have some adaptive measurement of the audible voice stream to measure the most dominant frequencies in the normal speech stream and compare packets so when a block of broad-spectrum data appears, attenuate it by some amount, like 50dB or so for as long as it's present. In other words, do some basic spectral analysis and isolate the speaker's dominant frequencies vs. those that may be missing most of the time. When you see a large increase in non-dominant broad-spectrum noise, assume it's a breath and attenuate the entire signal. People do not typically say anything meaningful when they're inhaling or exhaling, unless they're screaming very loudly, but that's an exhale. Usually this noise is most present during inhalation. When the device turns on or restarts, it needs to reinitialize its internal model by learning the speakers dominant frequencies in the audio stream for the current diver and adapt itself. So it's fine to hear the breath noises at first and hear them fade away after 10 breaths or so while the user is speaking normally. Perhaps have a short thing they need to read, like a poem, to help train the model. AI is not required! It just needs to be able to build a statistical model to differentiate the current speaker's normal speaking voice from the breath noise and suppress the sound when the breath noise is detected. It should go without saying, but ... divers won't swap out equipment while under water. So the same person will be using the device for perhaps several hours. It needs to keep a moving average, so occasional variations in the voice won't cause the breath noise to start showing up if the pitch of their voice changes somewhat under stress. Also, if helium is present in the air mix, the vocal pitch will increase, although this won't be a concern initially. For example: measure the presence of signals in, say, 20 frequency bands in the normal vocal range; some will be heavily present, and most will be almost empty / non-existent. When you see a noise that suddenly shows up in half or more of the bins that are normally empty, then attenuate that sample. (We'll probably need a way to adjust that threshold and perhaps other filter parameters easily during testing without recompiling.) A RPi Pico runs at 185 MHz or so, which should have plenty of bandwidth to handle vocals at a 22 kHz sampling rate in real-time. It can be developed in C/C++ or Pascal (Delphi, Free Pascal, etc) but it needs to be small enough to run in a RPi Pico or similarly configured device. This seems to me like a fairly simple task for someone familiar with FFTs and basic signal processing. I'm sure there are open source FFT libs and what really needs to be done is writing some code to sample frequencies and use it to recognize when a burst of white noise occurs on the input vs. normal speech patterns, and attenuate that signal in real-time. Here is a link that lets you DL the sample data file from Dropbox. REMOVE THE SPACES! ht tp : // w5a. com/u/ breath_sounds

FastMM is what I was referring to. I have not been able to get either of the two most recent V4 versions to work properly in Sidney. It works fine in Tokyo. I've brought it up here before, and apparently it's unique to my machine since nobody else seems to have a problem. I built a tool for it that parses out the log file and makes it super easy to figure out the likely source of the error. Most of the stuff in the log file is redundant, and it's mostly documenting side-effects of the error, not the error itself. Eg, failing to free the Objects in a stringlist throws a gazillion errors related to all of the orphaned objects, making it really hard to figure out that the problem was with the stringlist, which technically did nothing wrong. Also, in some situations, I'll get tons of orphaned string literals showing up that baffle me why they're even there. I forgot to properly free something that had a bunch of strings in it, and I guess Unicode sometimes gets them from the heap when they're added to certain containers (?). It's all just noise! In most cases, it comes down to missing one single call to .Free somewhere before either replacing something or freeing the container.

Nope. I spent a few hours trying to get those flow-things to work for my needs. They do something similar to word-wrapping, which does not preserve an X/Y grid layout which is what I was after. I could have used something like a big sheet of paper that slides around under a viewport and lets you shrink or expand the scale of the paper, but that's not what these things do. They do what they do. It's just not what I wanted.

I'm doing something similar right now. I found a TjanRoundedButton component somewhere and am using that. I lay them out on a panel in an x-by-y matrix. I ended up subclassing it to add some additional things I needed and I use instances of my own class on the panel. The original is fairly basic but easy to extend. I may ultimately switch to a grid, but to my eye, this approach looks better because it offers isolation between the "cells" whereas a grid jams them all together. In my case, I must create them at run-time because the number of rows and cols is data-driven. I found it easier to just lay them down algorithmically rather than use any of the fancy flowgrid/panel/etc options. I end up with one or more tabsheets, based on how many data sets are present. Then I add a TPanel on the tabsheet (which is probably redundant), and then add the buttons in a grid layout. Honestly, while prototyping I put them all down at design time, and it slowed the IDE down quite a bit. I didn't need them at design time, so doing it at run-time keeps the IDE from slowing down. (I've got this other thing that's a 9x9 array of radio buttons organized in columns using 9 panels, and it slows down the IDE as well. But they're static and it helps to have them at design time.)

Well, at this point I'm mostly in agreement with you. I haven't been able to get the memory manager to work in Sidney edition at all, and I normally use it to track down memory leaks. Without it, the best I can do is figure out where things are being created, but not where they're being freed or corrupted/overwritten unintentionally. This particular construct started out as a hack and it has been a problem from the start. I'm not sure why. But the in-memory table will be far more robust and less code overall, and I won't need to deal with dynamically created objects at all. I should have gone with it first. It can be hard when you go down the wrong path and feel so invested in it that you don't want to give it up. But this only getting worse and it's time to chuck it for something better.

Yup, it's just a bad approach that needs to be revised.

Uwe, that is ... UGLY! Not your fault, it just stinks. I do appreciate the effort. I'm going to try a couple of FDMemTables and DBGrids. So far it looks like it'll eliminate quite a bit of code, and just seems more natural and easier to manage.

Well, what I started out with was a list of one class, and there was just 1-3 records per student. Then I decided to combine classes, and the list got way too long. So I decided to split it into a sort of Master / Detail list where the top has a list of students, one line per student, with aggregate totals. When you click that, it displays all of the separate entries below for just that student. If you double-click, it takes those lower lines and sends them to another form to display them in a totally different way. So this code is aggregating all of the individual lines on a per-student basis and trying to put them into the upper list. I really don't care about what UI thing I use here. I just want a way to make it easier to manage what went from a fairly short list to a very long list when I combined all of this data together. What about an in-memory dataset and a TDBGrid?

Great idea! Unfortunately, compiler doesn't like it: Compiler says: [dcc32 Error] frmLSHM_Main.pas(1792): E2064 Left side cannot be assigned to Ditto here: procedure TListItemHelper.SetSnapTally(Value: ISnapTally); begin ISnapTally(Data) := Value; <-- Left side cannot be assigned to end; back to the same issue.

so copy it from the Delphi code, paste it into a new panel in Notepad++, do the work, then copy it and paste it over the old code in Delphi. I do it all the time to isolate chunks of code I want to change because its easier than having to keep restricting the bounds on what's replaced inside of the IDE. Or ... are you saying this needs to be done at run-time?

Which versions of Delphi does it work with?

If you have an older version of the code, then look through the DFMs with BeyondCompare or something similar and search for these component type names, then compare their properties.

Actually, I would encourage you to continue. It MUST be pursued! Because this is really the only way to get your mind to "flip" so you see things from an OOP perspective. It took me about 6 months of banging my head against these virtual walls and coding myself into dead-end alleys until one day I looked at the code and something flipped around and suddenly I saw it correctly. You've got to stick with it until that shift happens for you. You don't see it now. Once you do, it's game over for the "old" way of thinking. This is a perfect example to work with. Just keep banging away at it until you wrestle it to the ground, and step back and suddenly see what's required.

does it work with project groups as well?

I'm pretty sure you are struggling here because you don't have a clear idea of the notions of "encapsulation" when it comes to OOP. Or SOLID principles. Your GUI stuff should not need to be replaced with components created at run-time because THEY BELONG TO THE FORM and you are apparently unable to access them directly after removing this method from the form's class. So your solution is to back-door the form to override the encapsulation. You also said that after removing it you replaced all of the references to components on the form with T<theformname>, which is wrong. You want the INSTANCE variable name, not the TYPE. (Also, the TYPE is NOT the UNIT, which is I think what you're actually thinking of here.) These components are NOT static members of the form class (or the unit), they are MEMBERS of every INSTANCE of the FORM. So one way to solve this dilemma is to pass in a form parameter to the method that it needs to interact with, rather than give this unrelated function carte blanche to create stuff inside of the form's object! The problem with this approach is that then the email method is tightly coupled to this particular form. It also creates a circular reference between the UNITS. You could solve that by defining an Interface (in another unit) so the email method could be used with any form. But it's really not the job of the email method to update the UI anyway! It's job is to manage email, right? The FORM is responsible for interacting with the user. So you really have no business creating a way for the email method to even touch the UI. So scratch this one off as well. You need to realize that conceptually speaking, the email method could be stuck off on server somewhere and access as a REST service. In that case, the separation of responsibility becomes strikingly visible -- it's impossible for the email method to even reach the form, let alone update the UI. These are all classic symptoms that show up when someone doesn't get the gist of OOP "encapsulation" yet -- because your first inclination is to figure out ways around the encapsulation barriers, rather than thinking of how to restructure the interfaces to get you the results you desire. First, get the references to the UI OUT OF THE MAIL FUNCTION!!!! The UI should be managed EXCLUSIVELY by the FORM. If you want the function to update the UI, then pass in callback functions, or process things in a loop where you first do a setup, then call it for each update needed until it's done, then call a completion method. Read up on SOLID principles, especially SEPARATION OF RESPONSIBILITY. Better yet, pretend that this email method is on another server and all you can do is access it via a REST interface. That will get you the separation you need. It'll feel like you're standing on your head at first, but at least that's a sign that you're moving in the right direction. And if you find you need to split it into 3 or 4 different methods, so be it. And ... it probably belongs in its own class anyway that gets instantiated by the FORM to provide access to some email services. That is, it should live in a library and know absolutely nothing about who's using it.

I have a class that contains some logic that's applied to data selected by the user. The selections are added to a TListView, and an instance of the object is attached to the .Data property for each ListItem. They're being processed more or less in sequence right now, and I believe the overall processing time can be sped up a lot if I process each one as a task. I want to update columns in the ListView as each one finishes with a status message and elapsed time. But using the PPL, there's nothing that says when a task has completed. So I'm curous ... what's the best way to get an OnComplete event from each task to update the ListView and also figure out when everything has finished successfully?

I was able to do this with Omni Thread Library's Async method without too much trouble. I set up 20 dummy tasks that each took 2-14 seconds to run (randomly assigned). The entire batch took 14.8 seconds to run them all in parallel. THAT is what I was looking to accomplish! (For my application, I'll have 25-100 tasks, and most of them will be sitting waiting. If there's a problem, the timeout is 30 secs, but that rarely happens. The average I've seen is in the 10-15 second range. So if they can all get processed within 15-20 seconds, that'll be awesome.)

I've heard about the Omni Thread Library for years, authored by Primož Gabrijelcic, but have never more than glanced at it. Most of the examples I've seen have been fairly basic, and I didn't have any reason to suspect it might be more than what you find in Delphi's PPL. Boy, I couldn't have been more wrong! It took an amazing amount of digging around to find stuff, and after all of this I thought it would be helpful to memorialize my discoveries here for everybody to see. This is truly a remarkable piece of work. The library itself is FOSS, and Primož has posted stuff in his blog about it over time -- which spans 10 years now! It's a LOT of material to go through. You can get the latest source code (V3.07.9) on github here: https://github.com/gabr42/OmniThreadLibrary The version in GetIt is a bit outdated (3.07.7) Rather than publish a detailed Help guide or online resource, he chose to publish a book that can be purchased here: https://leanpub.com/omnithreadlibrary NOTE: you can get the book + the 3 webinars described later for even less than it says below at the LeanPub link. For the record, I don't particularly like digging through source code trying to figure things out, so I started hunting around for other resources. I found a recording (33 min) of CodeRage talk he gave at CodeRage 6 in 2011, and a free replay (MP4 download) of it can be found here: https://cc.embarcadero.com/item/28591 I watched that video and discovered that this is an UTTERLY AMAZING treasure trove of material, and it covers things I never imagined to see! It actually has seven (7) high-level abstract APIs (if you will) that really surprised me to see -- I'd figured it was mostly just the first few that are found in most threading libraries. * Async -- start a background task and continue -- this is similar to, yet more than what Delphi's PPL does * Future -- start background task, do some work, then wait to retrieve the result * ParallelTask -- start multiple copies of one task (for background procesing) and then wait for them all to complete * Join -- start several different background tasks and then wait for them to complete * Fork/Join -- do a divide and conquer, in parallel * ForEach -- perform a parallel iteration over an integer range, or on the contents of any container with built-in iterators (ie, that support for-in) * Pipeline -- run a multi-stage process I found a post here referring to an article using Deplhi's TCountdownEvent that has a bunch of code that does what Fork/Join does in just a handful of lines of code. What really blew my socks off were the ForEach and the Pipeline methods. I wanted more and kept digging.... eventually I found this: https://thedelphigeek.gumroad.com There you can find three hour-long tutorial videos on all of those high-level APIs. The link I found only went to the first video. I took a wild guess and found #3 that had the info on ForEach and Parallel, which is what I wanted, and bought that one. They're $10 each. Later I found out you can get all three for $25. ** EDIT: you can get these with the book at the LeanPub link above for just $5 more. NOTE: I'm mentioning this stuff here because it's so damn hard to find these things anywhere! You'll never find them searching for the term "video" because they're only referred to as "webinars". The couple of links I found on his blog specifically for these VIDEOS were dead and the ones in another place were wrong. I only found these because of one very brief and cryptic reply to a random question way down on one blog entry from 2015. I didn't notice them on LeanPub because I wasn't looking for "webinars". I started this search because I've been going around in circles trying to figure out the best way to deal with a collection of data displayed in a TListView with some data objects attached to the ListItems' .Data property, and I want to allow the user to hit a "Go!" button and have the code process the whole lot in parallel. They need to issue a bunch of REST service calls, so mostly they're just sitting idle waiting for replies, and I suspect they can all complete in the time it takes for the longest one to run. It turns out the ForEach<T> lets you iterate over the Listview's Items property using it's existing Iterator, and pass each TListItem into an anonymous Proc (by type!) and process it directly -- with each one running in parallel. You can easily do something when each of those threads finishes running, and also when the entire list is processed, all without having to send messages or go through crazy contortions to effect a fork/join operation. It's all handled invisibly. WHOA! The Pipeline abstraction is really something to behold. Many years ago I spent 9 months building a system that implemented a multi-step pipeline process that literally was spread out over a half-dozen different PCs on a LAN. Today they could all be done on one multi-core CPU, and using this bit of code would let me build it in less than a week (assuming all of the component parts were already working). In fact, i've built a few other systems that could have benefitted from this abstraction as well. And I can think of one coming up that might be able to use it. In summary ... maybe a lot of folks discovered this library when it was first published in 2010, but it seems to have virtually disappeared aside from an occasional mention in comparison to Delphi's Parallel Processing Library. Personally, I don't even think they're comparable. If you've never looked at it and are doing parallel programming stuff, take a look. And if you're famliar with it but may not have remembered all it does, give it another look and see if there's anything new worth studying. BTW, one thing that it had been criticized about was no way to Cancel threads in several places; the latest version (3.07.9) that's on github fixes that. It's relatively new, and not in the GetIt version.

Thanks, but it's not related. When I set up the Parallel.ForEach<TListItem>( aListView.Items ).Execute( procedure( const task: IOmniTask; const aLI : TListItem ) begin . . . I get this run-time error from the core logic trying to deal with the items it's getting -- clearly, they're TListItem type. First chance exception at $75CBB5B2. Exception class Exception with message 'TValue of type tkClass cannot be converted to TOmniValue'. Process ThreadingTest2VCL.exe (8516) The exception is being thrown in procedure OtlCommon.TOmniValue.SetAsTValue(const value: TValue); If I change it to a pointer (eg., Parallel.ForEach<pointer>), that fails as well, I'm guessing because the iterator (for TListView.Items) still returns an object that cannot be converted to a TOmniValue. I'm not sure if that's a bug or "as designed". I've found examples that show use of a TList<integer> that returns 'integer' types, and ForEach<TStringlist> that returns 'string' types. I haven't found anything that returns class instances, just basic types.

oops ... seems I was a bit overoptimistic with something.... I get a run-time error attempting to use iterators that return class types. The only examples I've found that work with collection classes, like TStringlist, all return basic types, like integers, strings, etc., not class instances. That seems rather odd because objects are merely references passed as typed pointers. Am I missing something here?

This is rather outdated and should be updated. And if they're going to seriously support IoT hardware like Raspberry Pi's and Arduios, they need to add Linux support into the Community Edition instead of forcing people to go with Free Pascal and Lazarus, because they run Linux on ARM CPUs. Now that all of Apples new computers run ARM CPUs, it's getting harder to ignore that option as well. And don't forget that MacOS is a full Unix system under the hood. Also, Windows is about the only platform that still supports 32-bit apps, although Raspian OS is working hard to get their 64-bit OS more stable so they can deprecate their 32-bit OS.

Perfect. Thanks!

But if class instances are attached to the .Data properties of the ListItems, then they'd need to know which LI they're attached to in order to update them, right? I'm thinking something like this to kick them all off: for var li in lview1.Items do begin TMyClass( li.Data ).Execute( li ); end; Would you create the task inside of the .Execute method? Or around that line? (suggesting that the 'li' parameter would probably not need to be injected) Would it make much of a difference? EDIT: actually, Omni Thread Library has a ForEach that looks like it can take the .Items directly and process them all as separate threads using the same Proc.

I prefer how Omni Thread Library does it: Go_btn.Enabled := False; Parallel.Async( procedure begin // executed in background thread Sleep(1000); end, Parallel.TaskConfig.OnTerminated( procedure (const task: IOmniTaskControl) begin // executed in main thread ShowMessage( 'done!' ); Go_btn.Enabled := True; end ) ); Right now, this bit of code is attached to a button.OnClick hander for testing. If this were inside of a class, I don't think the logic for updating the ListView belongs there. Which means that the OnTerminated proc there needs to trigger a common event handler, passing in an ID for which line / ListItem that thread is processing and perhaps a status indicator. Which is pretty much the same as using the queue you suggested. Or ... this whole bit of code is in a form method that's called and passed each ListItem where the .Data property has an object to be processed, and it has a method in it that the Async method runs as a task. Then the OnTerminated part updates the ListView. Which approach would you go with?

I'm looking more closely at the Omni Thread Library ... it seems to have more to offer than the what's in Delphi. Just no documentation or help files.