Best way to trigger in-app SoundFX supporting multiple audio devices and volume

[Yaron 60]

Delphi 7, Windows 10.

 

I wrote a menu system that plays SoundFX (basically WAV files) based on user actions.

I need to be able to control the playback device, volume and support asynchronous concurrent playback of multiple SFX.

 

To do this, I decided to go with DirectShow and it worked like a charm. However, each time you play an audio through DirectShow it creates multiple threads which are then terminated as the DirectShow graph is stopped.

So even if you play a single SFX, each time the SFX is triggered, Directshow will create and destroy several threads in the process.

I believe this behavior is contributing to memory fragmentation in my app.

 

Is there a method other than DirectShow to asynchronously play WAV files with volume control and allowing audio device selection for output?

 

Edited September 1, 2021 by Yaron

[Guest]

1 hour ago, Yaron said:

Is there a method other than DirectShow to asynchronously play WAV files with volume control and allowing audio device selection for output?

Yes there is, https://github.com/lakeofsoft/vcp

 

It can handle and play up to 32 channel concurrently, and it is high performant library and multithreaded out of the box, can adjust the volume however you want it ( file volume control or on the out device) , can handle multiple audio devices,  

DiectShow is fast and great but it is not meant to be used for small and short operation.

 

[Guest]

Forgot to mention that it does support scripting which might you find useful.

[Yaron 60]

@Kas Ob.

VCP looks like a massive library, I looked at the waveplayer demo and it doesn't seem to support output device selection.

 

If you have experience with VCP, can you give me a few pointers, my goal is play .wav files asynchronously in the background as the user is traversing the menu.

I currently use this function structure "PlaySound(sAudio : WideString; sDevice : String; iVolumeLevel : Integer);"

Where sAudio is the ".wav" filename, sDevice is the audio device name and iVolumeLevel is a volume value from 0 - 100.

 

Edited September 1, 2021 by Yaron

[Guest]

32 minutes ago, Yaron said:

I looked at the waveplayer demo and it doesn't seem to support output device selection.

All the components their support audio device selection, referenced by DeviceID, demonstrated in some demos, unless you meaning different thing about output device.

 

How many files are there ?

 

[Yaron 60]

@Kas Ob.

There are currently 5 events that trigger a different .wav file playback, might be extended to 9 in the future.

 

With regards to output devices, I mean on my PC I have speakers and a TV set, with DirectShow I can control whether the audio is played through the speakers or through the TV set.

[Guest]

Well, i am writing small demo to do this, only if my phone will stop ringing.

[Guest]

Well, spend two hours trying to find a bug !

 

There is a bug in loadFromStream, also digging more showed me that the library is not pooling its threads !, and i don't like that.

 

Anyway, this class should play and mix as much as you want, but still there is a rare bug somewhere with the mixer timing.

 

unit uVC_AsyncWavePlayer;

interface

uses
  Classes, SysUtils, unaVC_wave, unaVCIDE, unaVC_pipe, SyncObjs;//,unaMsAcmClasses;

type
  TWaveFile = class
  private
    FWaveFile: TunavclWaveRiff;
    FResampler: TunavclWaveResampler;
    FOnStreamIsDone: TNotifyEvent;
    FConsumer:unavclInOutPipe;
    procedure OnWaveStreamIsDone(sender: TObject);
  public
    constructor Create(Consumer: unavclInOutPipe);
    destructor Destroy; override;
    procedure PlaySound(const sAudio: string; iVolumeLevel: Integer);
    procedure Stop;
    property OnStreamIsDone: TNotifyEvent read FOnStreamIsDone write FOnStreamIsDone;
  end;

  TAsyncWavePlayer = class
  private
    FIdleList: TList;
    FAllList: TList;
    FLock: TCriticalSection;
    FWaveOut: TunavclWaveOutDevice;
    FWaveMixer: TunavclWaveMixer;
    FDeviceId: Integer;
    procedure OnWaveFileIsDone(sender: TObject);
    procedure SetDeviceId(const Value: Integer);
  protected
    function GetSoundPlayer: TWaveFile;
    procedure DisposePlayer(Player: TWaveFile);
  public
    constructor Create(DeviceId: Integer = -1);
    destructor Destroy; override;
    procedure PlaySound(const sAudio: string; iVolumeLevel: Integer);
    procedure Stop;
    property DeviceId: Integer read FDeviceId write SetDeviceId;
  end;

implementation

{ TWavePlayerCache }

constructor TAsyncWavePlayer.Create(DeviceId: Integer = -1);
begin
  FLock := TCriticalSection.Create;
  FIdleList := TList.Create;
  FAllList := TList.Create;

  FWaveOut := TunavclWaveOutDevice.Create(nil);
  FWaveOut.deviceId := DeviceId;
  FWaveOut.overNum := 64;

  //FWaveOut.waveEngine := unavcwe_ASIO;

  FWaveMixer := TunavclWaveMixer.Create(nil);
  FWaveMixer.isFormatProvider := True;
  FWaveMixer.realTime := True;
  FWaveMixer.consumer := FWaveOut;
  FWaveMixer.overNum := 64;
end;

destructor TAsyncWavePlayer.Destroy;
var
  I: Integer;
begin
  Stop;

  for I := 0 to FAllList.Count - 1 do
    TunavclWaveRiff(FAllList.Items[I]).Free;

  FIdleList.Free;
  FAllList.Free;
  FLock.Free;
  inherited;
end;

procedure TAsyncWavePlayer.DisposePlayer(Player: TWaveFile);
begin
  FLock.Acquire;
  try
    Player.Stop;
    if FIdleList.IndexOf(Player) = -1 then
      FIdleList.Add(Player);
  finally
    FLock.Release;
  end;
end;

function TAsyncWavePlayer.GetSoundPlayer: TWaveFile;
var
  Player: TWaveFile; {absolute Result;      // TWavePlayer  }
  Resampler: TunavclWaveResampler;
begin
  FLock.Acquire;
  try
    if FIdleList.Count > 0 then
    begin
      Result := FIdleList.Items[FIdleList.Count - 1];
      FIdleList.Remove(Result);
      Result.Stop;
    end
    else
    begin
      Player := TWaveFile.Create(FWaveMixer);
      Player.OnStreamIsDone := OnWaveFileIsDone;
      FAllList.Add(Player);
      Result := Player;
    end;
  finally
    FLock.Release;
  end;
end;

procedure TAsyncWavePlayer.OnWaveFileIsDone(sender: TObject);
var
  Player: TWaveFile absolute sender;
begin
  DisposePlayer(Player);
end;

procedure TAsyncWavePlayer.PlaySound(const sAudio: string; iVolumeLevel: Integer);
var
  Player: TWaveFile;
begin
  Player := GetSoundPlayer;
  Player.PlaySound(sAudio, iVolumeLevel);
end;

procedure TAsyncWavePlayer.SetDeviceId(const Value: Integer);
begin
  FDeviceId := Value;
  FWaveOut.deviceId := FDeviceId;
end;

procedure TAsyncWavePlayer.Stop;
var
  I: Integer;
begin
  FLock.Acquire;
  try
    for I := 0 to FAllList.Count - 1 do
      TWaveFile(FAllList[I]).Stop;
  finally
    FLock.Release;
  end;
end;

{ TWavePlayer }

constructor TWaveFile.Create(Consumer: unavclInOutPipe);
begin
  FConsumer :=Consumer;

  FWaveFile := TunavclWaveRiff.Create(nil);
  FWaveFile.onStreamIsDone := OnWaveStreamIsDone;
  //FWaveFile.autoCloseOnDone:=False;
  FWaveFile.isFormatProvider := True;
  FWaveFile.realTime := True;                  // Important for playing wave file

  FResampler := TunavclWaveResampler.Create(FWaveFile);
  FResampler.isFormatProvider := True;
  FResampler.overNum := 32;
  FResampler.realTime := True;

  FWaveFile.consumer := FResampler;
  //FResampler.consumer := FConsumer;
end;

destructor TWaveFile.Destroy;
begin
  Stop;
  FResampler.Free;
  FWaveFile.Free;
end;

procedure TWaveFile.OnWaveStreamIsDone(sender: TObject);
begin
  FResampler.consumer := nil;
  if Assigned(FOnStreamIsDone) then
    FOnStreamIsDone(Self);
end;

procedure TWaveFile.PlaySound(const sAudio: string; iVolumeLevel: Integer);
begin
  // may be there is a bug preventing this from working, also loadFromStream is not working
  // FWaveFile.loadFromFile(sAudio, False);
  FResampler.consumer := FConsumer;
  FResampler.setVolume100(iVolumeLevel);
  FWaveFile.fileName := sAudio;
  FWaveFile.open;
end;

procedure TWaveFile.Stop;
begin
  FWaveFile.close;
end;

end.

And use it like this

  WavePlayer:=TAsyncWavePlayer.Create(-1);
  // or
  WavePlayer:=TAsyncWavePlayer.Create;
  WavePlayer.DeviceId := 2;
  
  WavePlayer.PlaySound('BabyElephantWalk60.wav', 75);

To get the DeviceId and their names and indexes, please refer to the vMixerDemo

  f_mixerSystem := unaMsMixerSystem.create();
  if (0 < f_mixerSystem.getMixerCount()) then begin
    //
    i := 0;
    while (i < f_mixerSystem.getMixerCount()) do begin
      //
      f_mixerSystem.selectMixer(i);
      c_comboBox_mixerIndex.items.add(f_mixerSystem.getMixerName());
      inc(i);
    end;
    //
    c_comboBox_mixerIndex.itemIndex := 0;
  end;

 

 

ps : ASIO driver is also getting bugged with VC, and this indicate that something changed over years, may be even Windows 10 MSAcm has changed something that broke the library.

[Yaron 60]

@Kas Ob.

Thank you, I really appreciate your effort and time, I will try it out shortly.

 

How does the rare mixer timing bug manifest itself, what should I be looking for?

[Guest]

4 hours ago, Yaron said:

How does the rare mixer timing bug manifest itself, what should I be looking for?

I tested it with many files playing on random intervals (short intervals like 0.5-3 second while repeating this for tens of times) while the length of the wave files didn't matter, an AV raised sometimes, and strangely enough it happen only when the volume was higher than 100, when i repeated the test more than 20 times with volume <= 100 the AV didn't appear.

 

[Rollo62 563]

19 hours ago, Kas Ob. said:

Yes there is, https://github.com/lakeofsoft/vcp

Thanks for the info, that looks very interesting.

 

What is unclear to me, does it depend on Bass.dll ?

[Guest]

23 minutes ago, Rollo62 said:

What is unclear to me, does it depend on Bass.dll ?

No it doesn't depend on it, but it support it and can be used if you want, same for ASIO, but the library with ASIO in yesterday tests failed miserably. 

[Yaron 60]

@Kas Ob.  Passing a volume level greater than 100 does digital amplification? If that's the case then it's a non-issue as I don't plan on allowing greater than 100 volume.

[Guest]

1 hour ago, Yaron said:

If that's the case then it's a non-issue as I don't plan on allowing greater than 100 volume.

Well that is the problem, i can't say for sure it is the problem, VC is depending on MsACM driver to do many of its operation, also extend on them, and this AV is raised coming from a call back which will be very hard to catch red handed, so if you will go with that code above then just test it extensively.

 

On other hand there is another approach with simpler and will not fail, but will use more resources, see i used the mixer to limit the output to one device ( aka channel), but you can dedicate a shorter chain like Riff->WaveOut per file, by dropping (the somehow buggy or i am doing it wrong) mixer, also removing the resampler as you can configure each WaveOut format to WaveFile (Riff), this will simpler way, and even more simpler by unifying all the files format to one format then you don't need to resample or even check at runtime, keep in mind here that your sound card supports at least 64 output channels.

 

That was one approach, the other is to depend on one specific API PlaySound https://docs.microsoft.com/en-us/previous-versions/dd743680(v=vs.85) , this API also can do what you need, see the example

https://docs.microsoft.com/en-us/windows/win32/multimedia/playing-wave-resources?redirectedfrom=MSDN

Your wave files can be loaded form disk or memory or even the EXE resource.

The only thing is missing here which made me suggest VC to begin with is your need to change the Volume, now this can be solved by using VC to adjust the volume on the file or the streams in memory when you application started, or even test against doing it on the fly.

 

One thing to remember as it might not clear in documentation, realTime := True/False this control the pace of internal buffers to synchronize when playing in real time against do it as fast as you can, so if you went after changing the files volume without playing it using WaveOut then you need Riff->Resampler with both realTime set to false, and they will be blazing fast, if you want to save a file or a stream then make the chain like this Riff1->Resample->Riff2.

 

I hope that helps, and good luck.

[Guest]

Well forgot about the other thing other the volume, the output device !

 

This to work with PlaySound nicely you need to set the OutputDevice for the application which can be done, but searching landed me on either Russian or German resources, you need to use IPolicyConfig

that part of your code will look something like this

procedure TAudioControl.SetDefaultAudioDevice(devID: LPWSTR);
var
  pPolicyConfig: IPolicyConfig;
  res: HRESULT;
begin
  if CoCreateInstance(CLSID_PolicyConfig, nil, CLSCTX_ALL, IPolicyConfig,
    pPolicyConfig) = ERROR_SUCCESS then
  begin
    //pPolicyConfig.SetDefaultEndpoint(devID, 0);
    //pPolicyConfig.SetDefaultEndpoint(devID, 1);
    res := pPolicyConfig.SetDefaultEndpoint(devID, 2);
    if res = ERROR_SUCCESS then
    begin
            //pPolicyConfig.GetMixFormat;
    end else
    begin
    
    end;
  end;
end;

https://www.delphipraxis.net/169060-[open-source]-speedy-tray-audio-output-device-changer-nonvcl.html

http://www.delphikingdom.com/asp/answer.asp?IDAnswer=80887

 

Now you have many solution, like learn my half English, German, Russian or do it yourself !

 

[Yaron 60]

@Kas Ob.I wasn't intending for my app to do the mixing, windows does it itself automatically as long as the audio device is not locked exclusively.

 

I'm not sure if your solution is per-application or affects the entire system. I guess I have to do some more research.