Skullcode

Its really helpful And useful thank you very much 

On 8/29/2021 at 5:50 PM, Kas Ob. said:

Sorry, but it is amazing that it did made a sound !

 

So lets see what is wrong there

1) I don't know what input component/library you are using, but i can safely assume you know and can recognize in case your library is thread safe or not, this is not important for the following, but it is critical to remember when you fix the code above.

2) You usage of encoder and decoder is wrong, you are using them the way you use JPEG or PNG image encoder, and that is wrong, OPUS is encoder that continue encoding and refining based on every frame you feed it, while you are creating an encoder then encoding a frame then destroying it, you must create/destroy the encoder somewhere else, and in the handling/encoding part, you only can call opus_encode, same goes to the decoder.

3) You are encoding and decoding in the same event of capturing, meaning you slicing the time of the audio stream and introducing an interval that will sound bad, see if a frame 1/50 of second and the encoding and decoding took even 1/100 of second (assuming you had a decent CPU) then these 1/100 durations will either add silence durations like and the quality will drop and you might hear something like buzzer with the sound, this case with event based wave capturing (single threaded), or will cut out and will cause jumps and this will distort the audio and will sound like higher pitch or faster than normal, this is the case with background threading sound capture, in both case to minimize this effect you should consider moving the encoding/decoding from that event and put them in dedicated thread(s).

 

Now OPUS specifics, 

4) "cBITRATE = 500;"  ?? you should understand what bitrate stand for, it is the output in "bits per second", your 500bits is 63 byte per second, that way off, i would recommend 32000 as bitrate and that lets say the minimum decent at 4000 bytes per second.

5) "cSAMPLE_RATE = 16000;" Opus is adaptive, meaning you can go down but you can go up with sample rate and quality, but OPUS will do its better just like Speex at 48000 or 44100, it will be better quality even if you asking for lower bitrate.

6)  "cCHANNELS = 1;" well i didn't look for current version documentation, but i clearly remember that you are better with 2 channels, even if you are encoding mono mic input, because the sound card driver will supply the samples mono or stereo, OPUS will encode the data as it while obeying the bitrate, so when decoding there is no difference, 2 channels will ensure better quality, and will not cause you extra bitrate, and keep in mind this is the input and has nothing to do with the output, meaning if you asked OPUS for NarrowBand then it will output Mono !, also i would recommend WideBand at least.

7) "cFRAME_SIZE = 640;" Frame size is better at 1/50, meaning, for your 16000 it should be 320, this is recommended for real time application by the https://datatracker.ietf.org/doc/html/rfc6716#section-2.1.4 where 1/50 is 20ms.

 

may be 😎

I didn't like OPUS_APPLICATION_VOIP because it really get bad if there is noise around the speaker, i recommend to test OPUS_APPLICATION_AUDIO.

 

 

And good luck.

 

the component that i used to capture the microphone is http://www.lakeofsoft.com/

well try to create the encoder and the decoder outside of capture event

usually i should use the encoder outside of the capture event the code i posted just to demonstrate how i use the wrapper 

i use this wrapper to encode/decode 

https://github.com/DelphiForAudio/delphi-opus-wrapper/blob/master/uLibOpus.pas

here is what i am doing 

uses 
uLibOpus;

const
  cFRAME_SIZE = 640;
  cSAMPLE_RATE = 16000;
  cCHANNELS = 1;
  cAPPLICATION = OPUS_APPLICATION_VOIP;
  cBITRATE = 500;
  cMAX_PACKET_SIZE = 1280;
  
  
  procedure TForm1.recDataAvailable(sender: unavclInOutPipe; data: Pointer;
  len: Cardinal);
  var
  encoder: TOpusEncoder;
  decoder: TOpusDecoder;
  OpusEncoded : Tbytes;
  OpusDecoded : Tbytes;
  frame_size : integer;
  nbBytes : integer;
 begin
 
 
 if (data <> nil) and (len > 0) then
begin
//// Opus Encode
encoder := opus_encoder_create(cSAMPLE_RATE, cCHANNELS, cAPPLICATION, encoderError);
try
SetLength(OpusEncoded, cMAX_PACKET_SIZE);
nbBytes := opus_encode(encoder, data^, cFRAME_SIZE, Pointer(OpusEncoded)^, cMAX_PACKET_SIZE);

if nbBytes > 0 then
begin
SetLength(OpusEncoded, nbBytes);

//udRecive.Send(OpusEncoded, Length(OpusEncoded));
end;

finally
opus_encoder_destroy(encoder);
end;

//// End encoder Opus

////decoder opus
decoder := opus_decoder_create(cSAMPLE_RATE, cCHANNELS, encoderError);
try

SetLength(OpusDecoded, cMAX_PACKET_SIZE);


frame_size := opus_decode(decoder, Pointer(OpusEncoded)^, Length(OpusEncoded), Pointer(OpusDecoded)^, cFRAME_SIZE, 0);

SetLength(OpusDecoded, frame_size);

ply.write(OpusDecoded, length(OpusDecoded));

finally
opus_decoder_destroy(decoder);
end;

///  End decoder Opus
 
 end;

the decoded sound is very noisy and choppy i think i am missing something or doing something wrong

19 hours ago, Kas Ob. said:

Great to hear it is working, your are doing it right now, but still few things to reconsider.

 

1) Angus and Remy comments are very valuable and you really should read all the above and remember it, i missed to point you to the fact you always will be safer when using the working demo in any library as these demos/samples are done by the guys who know.

 

2) You supplied code, a working code "per your word" and still afraid if it is right, so i recommend that start experiment on your own, and please again if your are using ICS then test its demos and samples and try to understand how it is built, the only thing your will lose is your lack of confidence.

 

3) I pointed that you are better with 64k buffer for received, but as this subject does need many information, so i will explain a little again, and here i am assuming that you are familiar with UDP characteristics, are you ?

Are you sure you need UDP not TCP ?, (Google will help with these questions)

 

i will not go long on this as these information are on the internet, what i want to point is if you are not sure of how to receive UDP buffer then most likely you still don't know how to prevent lost packets in other word as you can't prevent it you need to manage and recover from it, with UDP there will be losing packets, and that depends on the data and your app (and many other factors out of your hand and control), in that case if the packets are critical then they need to be resend hence need a confirmation mechanism ,etc..

 

The point is read more, and the internet had many resources on this.

 

now back to 64k, you always better with receive buffer at 64k, always !, but this is not the case with send buffer !, so for receive with UDP i highly recommend to stick to 64k, while send buffer might be like Angus said 4k, why this is important

A) Smaller UDP packets size have lower rate of loss or dropped on the wire.

B) There is packets of all kind floating around, and your might need to test different approach, so by setting receive buffer maxed then you are forcing yourself to handle the data based on its content not its length, as this is very common mistake, also your app is ready for testing and tuning with different parameters in the future without the need to rebuild and redeploy both client and server, means you can tweak your system online or even can make it dynamic, like the more dropped packets, then lets the packets become smaller.

 

 

ps: i suggest in my code as best practice to use the buffer size as global const, and i don't understand the point of changing it to local var with fixed value, keeping such numbers as global constant will be better for tweaking in the future.

I am coming from a Java/Kotlin background using a UDP client in those platforms are different that's why I was really confused about receiving the event, the socket demo of ICS is very unorganized, for someone who doesn't have solid knowledge in Delphi As me, in other platforms a UDP client have a parameter in the receiving event That holds the bytes received and ready to use. I am reading each reply and I am trying to digest to understand better

14 hours ago, Kas Ob. said:

While waiting for someone with knowledge to answer your question, i want to point few things will help you not in this case only but for other cases.

So lets start with the IO operations, see, socket receiving and sending are Input/Output operations, hence they are out of control (your software) they mostly depends on the OS and the hardware, so as a rule of thumb you always should check for the result against your request, in UDP case and in socket in general you perform an operation with n bytes and all this methods should depends on OS methods and they always will return the m bytes ( m here is how did they managed to do) also m<n always, if you issued a send with 765 bytes then you should check if 765 bytes are sent, this is also true for files reading or writing... most IO operations are designed not designed to work as black and white or success or fail, but in many cases designed like "ok i managed to do 4 out of your 9"

so without in mined, your "but always returns 0" with your code does show how you do get that 0 ?!, and even in success reading you asked for 0 !, so your code wrong in at least two places and it should be like this

setLength(MBytes, SomeBufferSize);

BytesReceived := udRecive.Receive(MBytes, length(MBytes));
// SetLength(MBytes, BytesReceived); //We can perform this or not that is up to you
if BytesReceived > 0 then
  begin
      
  end else
  begin
  
  end;

But here how to decide SomeBufferSize , we can depends on IO operation to get some values, but in many case it is less efficient than ask for more or max we can handle and let the IO operation fill what it can, so is the best value for UDP buffer ? 

I Googled "maximum udp buffer size" and got a confirm 64k is the maximum UDP packet, the number is relatively small and manageable but any code, so why not to make it our standard buffer, so this code will be in general a better approach. 

const 
  OUR_MAX_UDP_BUFFER = 4 * 1024;
  
procedure TForm1.udReciveDataAvailable(Sender: TObject; ErrCode: Word);
var
  MBytes: TBytes;
begin
  setLength(MBytes, OUR_MAX_UDP_BUFFER);
  BytesReceived := udRecive.Receive(MBytes, length(MBytes));

  if BytesReceived > 0 then
    begin
    ...
    end;

also you can move that MBytes from local var to be a field on your Form1 hence you will need to allocate it once at biggest size like above and you don't need to trim it or free it.

Thank you very much for clarification i do something like this now 

var
OUR_MAX_UDP_BUFFER : integer;
MBytes: TBytes;
BytesReceived : integer;
begin

OUR_MAX_UDP_BUFFER := 4 * 1024;

setLength(MBytes, OUR_MAX_UDP_BUFFER);

BytesReceived := udRecive.Receive(MBytes, length(MBytes));


if BytesReceived > 0 then
begin
setLength(MBytes, BytesReceived);
//use the Mbytes as it needed

it is working but is what I am doing is correct? i am trying to handle it in a best way possible

I am trying to receive some bytes that I have sent.

but I am not able to specify the length of the byte since I do not know how to get the size of received bytes.

currently, I do  

procedure TForm1.udReciveDataAvailable(Sender: TObject; ErrCode: Word);
var
MBytes: TBytes;
begin

setLength(MBytes, 0);


udRecive.Receive(MBytes, length(MBytes));

but it always returns 0 length because I do set the length to 0 since I don't know how to set it to the exact length of the received Packet 

how do I know the exact length of the received packet to set it to the Mbytes variable?

I have found this wrapper for opus encoding and decoding 

https://github.com/DelphiForAudio/delphi-opus-wrapper

but I am confused on how to use it to encode/decode PCM audio data 

here is my PCM data coming from 

procedure TForm1.recDataAvailable(sender: unavclInOutPipe; data: Pointer;
  len: Cardinal);

begin

// I am not sure how to encode the data and its length using that wrapper 

// the example shows encoding decoding to wave files not directly to TBytes

end;

i use Twsocket as udp Client that is communicating with udpserver 

the server sends data in Bytes and the client should Receive This Data in Bytes as well

i have tried the following based on what i see in the Demo Sample of OverbyteIcsUdpLstn Demo

procedure TMainForm.WSocketDataAvailable(Sender: TObject; Error: Word);
var
aBytes : Tbytes;
begin
wsocket.Receive(aBytes, length(aBytes));
//...
end;

 but this freezes the application and immediately crashed

in my example i am stuck in sending those Bytes i am using Twsocket UDP 

that's what i do 

var
ABytes: TBytes;
EncodedBytes: TBytes;
bytesHead : TBytes;
baseaudio : string;
begin
baseaudio := 'Audio1'; 
SetLength(ABytes, len);
Move(data^, ABytes[0], len);
EncodedBytes := Encodeaudio(ABytes);

SetLength(bytesHead, 6 + Length(EncodedBytes));
TEncoding.UTF8.GetBytes(baseaudio, 1, 6, bytesHead, 0);
Move(EncodedBytes, bytesHead[6], Length(EncodedBytes));

wsocketudp.Send(bytesHead, Length(bytesHead))

when i send the data without adding header it sent just fine what is my Mistake ?

the exception raised 

First chance exception at $004072A3. Exception class $C0000005 with message 'access violation at 0x004072a3: read of address 0x0bd600d8'. Process room.exe (5680)

at this Line Move(EncodedBytes, bytesHead[6], Length(EncodedBytes));

i already Converted the RawToBytes To Tbytes 

as i posted up there 

var
Bytes: TBytes;

SetLength(Bytes, len);
Move(data^, Bytes[0], len);

i am confused to do the copying line 

CopyTIdString('Audio1', Bytes, 0);
CopyTIdBytes(RawToBytes(Buffer^, Buffersize), 0, Bytes, 6, Buffersize);

CopyTidstring requires TidBytes

i found this code in another forum 

var Bytes: TidBytes;

Bytes :=  RawToBytes(data^, len);

SetLength(Bytes, 6 + Buffersize);
CopyTIdString('Audio1', Bytes, 0);
CopyTIdBytes(RawToBytes(Buffer^, Buffersize), 0, Bytes, 6, Buffersize);

any idea how to convert it To  TBytes

i am not sure how to assign the Pointer data and do the exact same things with copying string using Tbytes  

var
Bytes: TBytes;

SetLength(Bytes, len);
Move(data^, Bytes[0], len);

From where this dots came up it is no in the string i posted very weird 

i use this Regex function to check if the string have a Characters that I specify in the pattern on not

Function CheckCharsinString(const astr: string):Boolean;
var
Regexs : TRegEx;
i : integer;
svalue : string;
Allowed : string;
begin

svalue := Trim(astr);

if Regexs.IsMatch(svalue, '^[ء-يA-Za-z0-9٠-٩⁰-⁹$™®©¤?@~<>«»✿❢•°،‘♔ہ¹ے;✧¢¯☆⊱٭✰❥¨.,’^*()%!\s-]+$') then
begin
Result := True;
end else
begin
Result := False;
end;

end;

when i check the following string  CheckCharsinString('✿شـ❢ـوكـ❢ـ‏‏ Dan✿') it always Return False.   

it should return True sense all that chars i use is in regex pattern . what is my Mistake ?

i am trying to build a voice communication app by sending a data from microphone to a Tcpserver and send it back to connected Clients 

i am looking for a component that can get a micrpohone audio input in bytes using opus codec in order to be able to send this data in real time .

is there any known library in delphi VCL that can capture audio input data that encoded to opus codec ?

Yes its very weird i guess its a bug with twebbrowser on reading similar gif names in multiple instances 

Any one tested and get same results? Is it a bug with twebbrowser itself? 

another example with Twebbrowser it cause same result the gif is included you can place it where the exe located 

webbrowser example.zip

this is a minimal example for better understanding 

webbrowser example.zip

i have created two Forms 

Form 1

Form 2

each form has TWebBrowser Control 

now i have a Gif that i call its name Foo.gif  

<img  src="Foo.gif" border="0" />

when i load this image in form1 TWebBrowser its animated normally .

when i load it in Form2 TWebBrowserit doesn't animated 

i am Adding this gif once at run Time To each form when form Created 

also i notice that this happened if the gif that i try to load to another TWebBrowser have the same name 

so if i load foo.gif in form1 it will animate in form1 hence when loaded to Form2 TWebBrowser and it got the same name of the previously loaded one it doesn't animate in Form2 TWebBrowser

any idea why this behavior happened  ?

can i have flashwindow effect to be applied on a VCL Tbutton Control ?

when i minimize My Vcl Application i do the following check in a Timer

if WindowState = wsMinimized  then
begin

//do some logging

end;

but this if condition doesn't detect whether the app is Minimized or not 

any idea how to detect if the app Minimized ?

is this a correct way to check if the Twsocket client is connected ?

if wsocket1 .State = wsConnected then
//do something 

i have a form with bsNone borderStyle when i execute 

WindowState := wsMaximized;

the taskbar became totally covered how can i avoid that ?

is there any way to read the microphone volume and sound output volume ?

in windows 10 the microphone level property is a range from 0 To 100 

also the sound level is a range from 0 to 100 

how can i read that ?

On 8/24/2020 at 3:30 PM, FPiette said:

I think since TWSocket exists for 24 years, if this bug exists, we should have already seen it. I think is is likely the code you wrote which is wrong. But I can tell...

 

1) Could you please show relevant code

2) How do you know that it is no a sever issue

3) Before writing you own code, check with one of the samples

 

The server is built with Java, i dont think that the issue within the server sense i already have another client in java works properly.

also tested in Delphi Tidtcpclient and it works Fine without missing packets. 

in Twsocket i am not doing anything special  , i have just placed Twsocket then ondataAvailable Event i read the data

procedure TForm1.asockDataAvailable(Sender: TObject; ErrCode: Word);
var
datastr : string;
begin
datastr := asock.ReceiveStrW(CP_UTF8);
Memo1.lines.add(datastr);
end;

The server is using CRLF for each packet maybe this is an issue ?