encoding pcm to opus and decoding

[Skullcode 0]

i use this wrapper to encode/decode 

 

https://github.com/DelphiForAudio/delphi-opus-wrapper/blob/master/uLibOpus.pas

 

 

here is what i am doing 

 

 

uses 
uLibOpus;

const
  cFRAME_SIZE = 640;
  cSAMPLE_RATE = 16000;
  cCHANNELS = 1;
  cAPPLICATION = OPUS_APPLICATION_VOIP;
  cBITRATE = 500;
  cMAX_PACKET_SIZE = 1280;
  
  
  procedure TForm1.recDataAvailable(sender: unavclInOutPipe; data: Pointer;
  len: Cardinal);
  var
  encoder: TOpusEncoder;
  decoder: TOpusDecoder;
  OpusEncoded : Tbytes;
  OpusDecoded : Tbytes;
  frame_size : integer;
  nbBytes : integer;
 begin
 
 
 if (data <> nil) and (len > 0) then
begin
//// Opus Encode
encoder := opus_encoder_create(cSAMPLE_RATE, cCHANNELS, cAPPLICATION, encoderError);
try
SetLength(OpusEncoded, cMAX_PACKET_SIZE);
nbBytes := opus_encode(encoder, data^, cFRAME_SIZE, Pointer(OpusEncoded)^, cMAX_PACKET_SIZE);

if nbBytes > 0 then
begin
SetLength(OpusEncoded, nbBytes);

//udRecive.Send(OpusEncoded, Length(OpusEncoded));
end;

finally
opus_encoder_destroy(encoder);
end;

//// End encoder Opus

////decoder opus
decoder := opus_decoder_create(cSAMPLE_RATE, cCHANNELS, encoderError);
try

SetLength(OpusDecoded, cMAX_PACKET_SIZE);


frame_size := opus_decode(decoder, Pointer(OpusEncoded)^, Length(OpusEncoded), Pointer(OpusDecoded)^, cFRAME_SIZE, 0);

SetLength(OpusDecoded, frame_size);

ply.write(OpusDecoded, length(OpusDecoded));

finally
opus_decoder_destroy(decoder);
end;

///  End decoder Opus
 
 end;

the decoded sound is very noisy and choppy i think i am missing something or doing something wrong

[Guest]

1 hour ago, Skullcode said:

the decoded sound is very noisy and choppy i think i am missing something or doing something wrong

Sorry, but it is amazing that it did made a sound !

 

So lets see what is wrong there

1) I don't know what input component/library you are using, but i can safely assume you know and can recognize in case your library is thread safe or not, this is not important for the following, but it is critical to remember when you fix the code above.

2) You usage of encoder and decoder is wrong, you are using them the way you use JPEG or PNG image encoder, and that is wrong, OPUS is encoder that continue encoding and refining based on every frame you feed it, while you are creating an encoder then encoding a frame then destroying it, you must create/destroy the encoder somewhere else, and in the handling/encoding part, you only can call opus_encode, same goes to the decoder.

3) You are encoding and decoding in the same event of capturing, meaning you slicing the time of the audio stream and introducing an interval that will sound bad, see if a frame 1/50 of second and the encoding and decoding took even 1/100 of second (assuming you had a decent CPU) then these 1/100 durations will either add silence durations like and the quality will drop and you might hear something like buzzer with the sound, this case with event based wave capturing (single threaded), or will cut out and will cause jumps and this will distort the audio and will sound like higher pitch or faster than normal, this is the case with background threading sound capture, in both case to minimize this effect you should consider moving the encoding/decoding from that event and put them in dedicated thread(s).

 

Now OPUS specifics, 

4) "cBITRATE = 500;"  ?? you should understand what bitrate stand for, it is the output in "bits per second", your 500bits is 63 byte per second, that way off, i would recommend 32000 as bitrate and that lets say the minimum decent at 4000 bytes per second.

5) "cSAMPLE_RATE = 16000;" Opus is adaptive, meaning you can go down but you can go up with sample rate and quality, but OPUS will do its better just like Speex at 48000 or 44100, it will be better quality even if you asking for lower bitrate.

6)  "cCHANNELS = 1;" well i didn't look for current version documentation, but i clearly remember that you are better with 2 channels, even if you are encoding mono mic input, because the sound card driver will supply the samples mono or stereo, OPUS will encode the data as it while obeying the bitrate, so when decoding there is no difference, 2 channels will ensure better quality, and will not cause you extra bitrate, and keep in mind this is the input and has nothing to do with the output, meaning if you asked OPUS for NarrowBand then it will output Mono !, also i would recommend WideBand at least.

7) "cFRAME_SIZE = 640;" Frame size is better at 1/50, meaning, for your 16000 it should be 320, this is recommended for real time application by the https://datatracker.ietf.org/doc/html/rfc6716#section-2.1.4 where 1/50 is 20ms.

 

may be 😎

I didn't like OPUS_APPLICATION_VOIP because it really get bad if there is noise around the speaker, i recommend to test OPUS_APPLICATION_AUDIO.

 

 

And good luck.

 

[Skullcode 0]

On 8/29/2021 at 5:50 PM, Kas Ob. said:

Sorry, but it is amazing that it did made a sound !

 

So lets see what is wrong there

1) I don't know what input component/library you are using, but i can safely assume you know and can recognize in case your library is thread safe or not, this is not important for the following, but it is critical to remember when you fix the code above.

2) You usage of encoder and decoder is wrong, you are using them the way you use JPEG or PNG image encoder, and that is wrong, OPUS is encoder that continue encoding and refining based on every frame you feed it, while you are creating an encoder then encoding a frame then destroying it, you must create/destroy the encoder somewhere else, and in the handling/encoding part, you only can call opus_encode, same goes to the decoder.

3) You are encoding and decoding in the same event of capturing, meaning you slicing the time of the audio stream and introducing an interval that will sound bad, see if a frame 1/50 of second and the encoding and decoding took even 1/100 of second (assuming you had a decent CPU) then these 1/100 durations will either add silence durations like and the quality will drop and you might hear something like buzzer with the sound, this case with event based wave capturing (single threaded), or will cut out and will cause jumps and this will distort the audio and will sound like higher pitch or faster than normal, this is the case with background threading sound capture, in both case to minimize this effect you should consider moving the encoding/decoding from that event and put them in dedicated thread(s).

 

Now OPUS specifics, 

4) "cBITRATE = 500;"  ?? you should understand what bitrate stand for, it is the output in "bits per second", your 500bits is 63 byte per second, that way off, i would recommend 32000 as bitrate and that lets say the minimum decent at 4000 bytes per second.

5) "cSAMPLE_RATE = 16000;" Opus is adaptive, meaning you can go down but you can go up with sample rate and quality, but OPUS will do its better just like Speex at 48000 or 44100, it will be better quality even if you asking for lower bitrate.

6)  "cCHANNELS = 1;" well i didn't look for current version documentation, but i clearly remember that you are better with 2 channels, even if you are encoding mono mic input, because the sound card driver will supply the samples mono or stereo, OPUS will encode the data as it while obeying the bitrate, so when decoding there is no difference, 2 channels will ensure better quality, and will not cause you extra bitrate, and keep in mind this is the input and has nothing to do with the output, meaning if you asked OPUS for NarrowBand then it will output Mono !, also i would recommend WideBand at least.

7) "cFRAME_SIZE = 640;" Frame size is better at 1/50, meaning, for your 16000 it should be 320, this is recommended for real time application by the https://datatracker.ietf.org/doc/html/rfc6716#section-2.1.4 where 1/50 is 20ms.

 

may be 😎

I didn't like OPUS_APPLICATION_VOIP because it really get bad if there is noise around the speaker, i recommend to test OPUS_APPLICATION_AUDIO.

 

 

And good luck.

 

the component that i used to capture the microphone is http://www.lakeofsoft.com/

 

well try to create the encoder and the decoder outside of capture event

usually i should use the encoder outside of the capture event the code i posted just to demonstrate how i use the wrapper 

Edited August 30, 2021 by Skullcode

[Guest]

@Skullcode I don't know if you have working solution now, just tried to use opus for fun now this morning, and i found that i missed something in your code, and your code is doing it wrong.

The problem is clear when you check the documentation of the source code of OPUS library, opus_decode return sample count not bytes ! and this in need to be adjusted to bytes before passing it out to WaveOut device.

So i made a small application to test the codec in loopback (mic-speaker) and compare against the un-encoded, and hope you find it clear and useful.

VC_Opus.zip
VC_Opus_DLL32.zip

[Skullcode 0]

Its really helpful And useful thank you very much