Indy HTTP server

Joe Sansalone · August 8, 2021

(Currently trying with updated openssl DLL files to see if it fixes the problem .. although I need to wait a while to know for sure)

In the meantime ...

Hi,

When testing a Indy Server application everything was fine.

Now in production, it stops responding to requests after a while.

Is it ok to bind multiple ports like this? (should bind to the same default IP, right?)

if not FServer.Active then
begin
FServer.Bindings.Clear;
FServer.Bindings.Add.Port := 443;
FServer.DefaultPort := StrToInt(EditPort.Text);
FServer.Bindings.Add.Port := 80;
FServer.Bindings.Add.Port := 8080; // for testing
FServer.Active := True;
end;

Another question:

If an exception occurs in the OnCommandGet of the server and is not handled, would this screw things up for Indy HTTP server?

Am I fine as long as I set this before end of OnCommandGet?

AResponseInfo.ResponseNo := 200;
AResponseInfo.ContentText := 'OK'; // or whatever appropriate response here
AResponseInfo.ContentType := 'text/html;charset="UTF-8"';

What else could I be doing to get IdHTTPServer to stop responding to requests?

Thanks,

Joe

Edited August 8, 2021 by Joe Sansalone

Remy Lebeau · August 8, 2021

2 hours ago, Joe Sansalone said:

When testing a Indy Server application everything was fine.

Now in production, it stops responding to requests after a while.

That has nothing to do with the ports you are using, unless you have a firewall that is blocking them. Which version of Indy and OpenSSL DLLs are you using?

Quote

Is it ok to bind multiple ports like this? (should bind to the same default IP, right?)

Yes. Though, you don't need to set the DefaultPort at all if you are just going to set each Binding.Port explicitly. If you are going to bind to multiple ports, then the DefaultPort should be 0. Just make sure you are handling the OnQuerySSLPort event correctly, especially for non-standard ports, like 8080. Also, make sure you are using an up-to-date version of Indy, since the behavior of default HTTPS handling did change a few years ago.

Quote

If an exception occurs in the OnCommandGet of the server and is not handled, would this screw things up for Indy HTTP server?

No, that is perfectly fine, even expected and intentionally designed for. You SHOULD be throwing exceptions on unexpected failures. The server will catch and process uncaught exceptions internally. For socket I/O errors, it will close the socket and stop the socket's owning thread, triggering OnDisconnect and OnException events. For most other exceptions, it will instead trigger the OnCommandError event, then send an error response to the client if a response has not already been sent, and then carry on with normal HTTP KeepAlive handling - closing the socket or waiting for the next HTTP request, as needed.

Quote

Am I fine as long as I set this before end of OnCommandGet?

If an exception is raised and not caught, the server will catch it, and in most cases will overwrite those values with its own error values.

Quote

What else could I be doing to get IdHTTPServer to stop responding to requests?

There is no way to answer that with the limited information you have provided. Do you have KeepAlive enabled or disabled on the server? Once the server stops responding, are you getting OnConnect events for new connections, and getting OnHeadersAvailable events for new requests, at least? What do your OnCommand... events actually look like? Have you verified with a packet sniffer like Wireshark that your server machine is still receiving new HTTP requests from the network?

Edited August 8, 2021 by Remy Lebeau

Joe Sansalone · August 9, 2021

I'm using Indy 10.6.2.0.

The OpenSSL DLLs were more than a year old, so I just updated the DLLs today with 1.0.2u version (Jun 25, 2021).

I'm hoping these new DLLs might solve the problem - I later learned that Twilio server error logs indicated that a problem occurred

during the TLS handshaking. Twilio servers send requests to my HTTP(S) application server.

I am using the OnQuerySSLPort event. I only return True (use SSL) for the 443 port.

I didn't get a chance to use Wireshark when the problem happened.

If it happens again, I will.

My OnCommand events do not create any object. They use pre-created objects

with pre-connected database connections to handle requests. I was careful to make

sure the code is thread-safe. In general, all requests are handled by separate threads

anyway (pooled).

Thanks for pointing out OnConnect, OnHeadersAvailable events ...

I'll put some code there to log info so that the next time it happens, I'll see more.

Also, OnException/OnCommandError ... I'll put some logging code.

Thanks for your help.

I will write again - either saying it's fixed OR if it happens again

I'll update this thread with more logging information.

Joe

Remy Lebeau · August 9, 2021

1 hour ago, Joe Sansalone said:

I'm using Indy 10.6.2.0.

Which specific release of 10.6.2 exactly? Are you using a stock version that shipped pre-installed in a specific IDE version? Or have you updated to the latest trunk version from Indy's GitHub repo? Indy's versioning has been broken since Indy switched from SVN to GitHub, so 10.6.2.0 does not accurately reflect the real version.

1 hour ago, Joe Sansalone said:

The OpenSSL DLLs were more than a year old, so I just updated the DLLs today with 1.0.2u version (Jun 25, 2021).

That should be OK.

1 hour ago, Joe Sansalone said:

I'm hoping these new DLLs might solve the problem - I later learned that Twilio server error logs indicated that a problem occurred

during the TLS handshaking.

Any specific details about the errors?

1 hour ago, Joe Sansalone said:

I am using the OnQuerySSLPort event. I only return True (use SSL) for the 443 port.

OK.

1 hour ago, Joe Sansalone said:

My OnCommand events do not create any object. They use pre-created objects

with pre-connected database connections to handle requests. I was careful to make

sure the code is thread-safe. In general, all requests are handled by separate threads

anyway (pooled).

Any synchronization between threads being done?

Joe Sansalone · August 9, 2021

I'm using Indy 10.6.2.0 that came with Delphi 10.4.2.

Details on error from Twilio:

SSL/TLS Handshake Error

An attempt to retrieve content from https://live.projectone.ca/Phone returned the HTTP status code 502

During SSL/TLS negotiation, Twilio experienced a connection reset.

12 hours ago, Remy Lebeau said:

Any synchronization between threads being done?

Yes, there's some synchronization. I'm pretty sure I put some timeouts to make sure

that it returns. You bring up a good point. I'm assuming you are suggesting that

synchronization can cause the OnCommand events to timeout, correct? Which could

be a problem.

Remy Lebeau · August 9, 2021

1 hour ago, Joe Sansalone said:

I'm using Indy 10.6.2.0 that came with Delphi 10.4.2.

OK.

Quote

Details on error from Twilio:

SSL/TLS Handshake Error

An attempt to retrieve content from https://live.projectone.ca/Phone returned the HTTP status code 502

Makes sense, if Twilio sends a request to ProjectOne, which sends a request to your server, which fails, then ProjectOne would send an error back to Twilio.

Quote

During SSL/TLS negotiation, Twilio experienced a connection reset.

Are you sure it is Twilio and not ProjectOne? In any case, a connection reset during a TLS handshake usually means the server did not like something in the client's (Twilio/ProjectOne) handshake data, so it simply chose to abort the connection rather than send a TLS alert back to the client explaining the reason.

Quote

Yes, there's some synchronization. I'm pretty sure I put some timeouts to make sure

that it returns. You bring up a good point. I'm assuming you are suggesting that

synchronization can cause the OnCommand events to timeout, correct?

Hang/deadlock rather than timeout, but yes, if you are not careful with it.

Edited August 9, 2021 by Remy Lebeau

Joe Sansalone · August 9, 2021

40 minutes ago, Remy Lebeau said:

Hang/deadlock rather than timeout, but yes, if you are not careful with it.

I'll review my code looking for possible problems with synchronization (although I remember being careful when writing it).

The production application now has logging on OnException, OnCommendError, OnHeadersAvailable, OnConnect.

And using the updated SSL DLLs.

So I'll wait and see if it happens again.

Joe Sansalone · August 10, 2021

Now that logging is enabled on the OnException event handler of TIdHTTPServer,

I'm seeing all sorts of different exceptions.

However, the application is responding correctly.

These are other "bad" requests hitting the server.

I guess this is normal?

Below is part of the log:

[HTTP : Exception 08/10 07:01:51.935] OnException: Error accepting connection with SSL.
EOF was observed that violates the protocol

[HTTP : Exception 08/10 07:45:07.863] OnException: Socket Error # 10054
Connection reset by peer.
[HTTP : Exception 08/10 07:45:08.035] OnException: Socket Error # 10054
Connection reset by peer.
[HTTP : Exception 08/10 07:45:08.113] OnException: Error accepting connection with SSL.
error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher
[HTTP : Exception 08/10 07:45:08.379] OnException: Socket Error # 10054
Connection reset by peer.
[HTTP : Exception 08/10 07:45:08.582] OnException: Socket Error # 10054
Connection reset by peer.
[HTTP : Exception 08/10 07:45:08.738] OnException: Socket Error # 10054
Connection reset by peer.
[HTTP : Exception 08/10 07:45:08.926] OnException: Socket Error # 10054
Connection reset by peer.
[HTTP : Exception 08/10 07:45:09.113] OnException: Socket Error # 10054
Connection reset by peer.
[HTTP : Exception 08/10 07:45:09.348] OnException: Socket Error # 10054
Connection reset by peer.
[HTTP : Exception 08/10 07:45:09.567] OnException: Socket Error # 10054
Connection reset by peer.
[HTTP : Exception 08/10 07:45:29.097] OnException: Connection Closed Gracefully.
[HTTP : Exception 08/10 08:05:17.495] OnException: Connection Closed Gracefully.
[HTTP : Exception 08/10 08:33:29.720] OnException: Error accepting connection with SSL.
error:1408A10B:SSL routines:ssl3_get_client_hello:wrong version number
[HTTP : Exception 08/10 09:01:26.542] OnException: Error accepting connection with SSL.
EOF was observed that violates the protocol
[HTTP : Exception 08/10 09:41:58.993] OnException: Error accepting connection with SSL.
error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
[HTTP : Exception 08/10 09:57:41.154] OnException: Connection Closed Gracefully.

Joe Sansalone · August 17, 2021

The application (HTTP Server) stopped responding to requests again.

This time, I simply "Stopped" and "Started" the TIdHTTPServer without restarting the application and it worked - application started

responding to requests.

// Stopped

procedure TForm1.ButtonStopClick(Sender: TObject);
begin
FServer.Active := False;
FServer.Bindings.Clear;
end;

// Started

procedure TForm1.StartServer;
begin
if not FServer.Active then
begin
FServer.Bindings.Clear;
FServer.Bindings.Add.Port := 443;
FServer.DefaultPort := StrToInt(EditPort.Text);
FServer.Bindings.Add.Port := 80;
FServer.Bindings.Add.Port := 8080; // for testing
FServer.Active := True;
end;
end;

I'm either doing something strange in the TIdHTTPServer events or Indy HttpServer has a bug.

BELOW is my code.

It's just a bunch of "If then begin end" to handle each Pathinfo differently.

And all of it is wrapped in a Try/Except

procedure TForm1.FServerCommandGet(AContext: TIdContext;
ARequestInfo: TIdHTTPRequestInfo; AResponseInfo: TIdHTTPResponseInfo);
var
// engine: TSMSEngineIn;
DB: TDynamicDBSession;
task, numMedia, numSegments, profile_id: integer;
PathInfo, id, temp, DID, NoResponse, callerID, main, mapped, PathInfoUser: string;
ms: TMemoryStream;
flowEngine: TFlowEngine;
sms: TTwilioSMS;
nexmo: TNexmoSMS;
ibm: TIBMTranslator;
from, toDN, text, forwardedFrom: string;
FLog: TLogger;
MsgStatus, InParams, tag, ip: string;

CallSid, TwiMLResponse, engineID, CallStatus, SpeechResult, StableSpeechResult, UnstableSpeechResult,
seqNumber, DigitsResult, Confidence: string;

DialCallSid, DialCallStatus, TwilioPayResult, TwilioPayError, QueueResult, QueueTime, DequeingCallSid: string;

RecordingSid, RecordingUrl, SMSSid: string;
num, address: string; // form input from web
isMobile, privatecall, isOptedOut: boolean;
begin
PathInfo := ARequestInfo.URI;
PathInfoUser := UpperCase(PathInfo); // when it's a user typing url in a browser, we want to make sure they can spell it
// upper, lower etc

try
NoResponse := '<?xml version="1.0" encoding="UTF-8"?> <Response> </Response>';
AResponseInfo.ContentType := 'text/html;charset="UTF-8"';

// if incoming params are empty, it's a "fake" request
InParams := ARequestInfo.Params.DelimitedText;

// default Path
if ((PathInfo = '/') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: path /');
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'Params: ' + InParams);

From := RemovePlus1(aRequestInfo.Params.Values['From']);
ToDN := RemovePlus1(aRequestInfo.Params.Values['To']);

// avoid hackers

if ((From = '') and (ToDN='')) then
begin
AResponseInfo.ResponseNo := 404;
AResponseInfo.ResponseText := 'Bug off';
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'IP: ' + AContext.Binding.PeerIP);
Exit;
end;

Text := IndyTextEncoding_UTF8.GetString(ToBytes(aRequestInfo.Params.Values['Body'], IndyTextEncoding_8Bit));

DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'From: ' + From + ' To: ' + ToDN);
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'Text: ' +StringReplace(Text, #13#10, ' ', [rfReplaceAll]));

FDisplayThread.Display(TimeToStr(Now) + ': SMS From: ' + From + ' To: ' + ToDN);
FDisplayThread.Display(TimeToStr(Now) + ': Text: ' + Text);

try
NumMedia := StrToInt(aRequestInfo.Params.Values['NumMedia']);
numSegments := StrToInt(aRequestInfo.Params.Values['NumSegments']);
except on E:EConvertError do
begin
NumMedia := 0;
numSegments := 1;
end;
end;
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'MMS: ' + IntToStr(numMedia));

// TODO: 1st check to see if session is already active

if EngineSessions.LookupSession(From + toDN, task) then
begin
EnginePool[task].Engine.SMSEvent(From, toDN, Text, NumMedia, numSegments);
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'SMSEvent');
end
else // new
if EnginePool.GetEngineFromPool(task) then
begin
EngineSessions.AddSession(From + toDN, task);
EnginePool[task].IncomingSMS(toDN, From, Text, NumMedia, numSegments,
aRequestInfo.Params.Values['MediaUrl0'], // 1st Media URL for now
aRequestInfo.Params.Values['MediaContentType0']);
AResponseInfo.ContentText := NoResponse;
end
else
begin
DefaultLog.WriteTimeStamp(lctSMS, lptImportant, 'Unable to get an Engine from Pool');
AResponseInfo.ContentText := TTwilioML.NoCapacitySMS; // for get this
// AResponseInfo.ContentText := 'Overloaded server';
// AResponseInfo.ResponseNo := 503; // busy, overloaded, not available for now
end;

AResponseInfo.ContentType := 'text/html;charset="UTF-8"'; // IMPORTANT to have no spaces in ContentType
end;

// Nexmo SMS
if ((PathInfo = '/Nexmo/SMS') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '**** Begining webmodule default action: path /Nexmo/SMS');
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'Params: ' + InParams);

From := Remove1(aRequestInfo.Params.Values['msisdn']);
ToDN := Remove1(aRequestInfo.Params.Values['to']);
Text := IndyTextEncoding_UTF8.GetString(ToBytes(aRequestInfo.Params.Values['text'], IndyTextEncoding_8Bit));

DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'From: ' + From + ' To: ' + ToDN);
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'Text: ' +StringReplace(Text, #13#10, ' ', [rfReplaceAll]));

// 1st check to see if session is already active
// not needed for SMS at this time

if EnginePool.GetEngineFromPool(task) then
begin
EngineSessions.AddSession(From + toDN, task);
EnginePool[task].IncomingSMS(toDN, From, Text, 0, 1, '', ''); // no media stuff for nexmo

end
else
DefaultLog.WriteTimeStamp(lctSMS, lptImportant, 'Unable to get an Engine from Pool');

AResponseInfo.ResponseNo := 200;
AResponseInfo.ContentText := 'OK';
AResponseInfo.ContentType := 'text/html;charset="UTF-8"'; // IMPORTANT to have no spaces in ContentType

end;

// Nexmo Delivery Receipt
if ((PathInfo = '/Nexmo/DLR') and (InParams <> '')) then
begin
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Begining webmodule default action: path /Nexmo/DLR');
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'Params: ' + InParams + #13#10);

if (aRequestInfo.Params.Values['status'] <> 'delivered') then
begin
DefaultLog.WriteTimeStamp(lctSMS, lptImportant, 'Nexmo msg not delivered');
end;

AResponseInfo.ResponseNo := 200;
AResponseInfo.ContentText := 'OK';
AResponseInfo.ContentType := 'text/html;charset="UTF-8"'; // IMPORTANT to have no spaces in ContentType
end;

// Twilio Delivery Receipt
if ((PathInfo = '/Status') and (InParams <> '')) then
begin
// DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'Params: ' + InParams + #13#10);
MsgStatus := ARequestInfo.Params.Values['MessageStatus'];
//if ((MsgStatus <> 'sent') and (MsgStatus <> 'delivered')) then
if (MsgStatus = 'delivered') then
begin
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, '***** /Status delivered' );
SMSSid := ARequestInfo.Params.Values['SMSSid'];
if EngineSessions.LookupSession(SMSSid, task) then
begin
EnginePool[task].Engine.SignalSMSDelivery;
end

end;

AResponseInfo.ContentText := NoResponse;
AResponseInfo.ContentType := 'text/html;charset="UTF-8"'; // IMPORTANT to have no spaces in ContentType
end;

// Twilio CallStatus events
if ((PathInfo = '/CallStatus') and (InParams <> '')) then
begin

DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** /CallStatus ');
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Params: ' + InParams + #13#10);

CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid + ' sessions cnt:' + IntToStr(EngineSessions.Count) );
CallStatus := aRequestInfo.Params.Values['CallStatus'];
if (CallStatus = 'completed') or (CallStatus = 'canceled') then
begin
if EngineSessions.LookupSession(CallSid, task) then
begin
EngineSessions.RemoveSession(CallSid);
EnginePool[task].Engine.SignalHangup;
//send hangup event
end
else
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** /CallStatus: ' + 'did not find a session ');
end
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '***** /CallStatus: ' + CallStatus + ' something other than completed/canceled??');

AResponseInfo.ContentText := NoResponse;
AResponseInfo.ContentType := 'text/html;charset="UTF-8"'; // IMPORTANT to have no spaces in ContentType
end;

// Twilio MakeCallStatus events
// we send the engine# via the url, so no need to lookup session
// however, in future, we may need to lookup session to get OTHER engines that are interested in this status
if ((PathInfo = '/MakeCallStatus') and (InParams <> '')) then
begin

DefaultLog.WriteTimeStamp(lctSMS, lptNormal, '***** /MakeCallStatus ');
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'Params: ' + InParams + #13#10);

CallSid := aRequestInfo.Params.Values['CallSid'];
engineID := aRequestInfo.Params.Values['engineid'];
task := StrToInt(engineID);
CallStatus := aRequestInfo.Params.Values['CallStatus'];

if (CallStatus = 'completed') then // i.e. hungup after connected
begin
if EngineSessions.LookupSession(CallSid, task) then
begin
EngineSessions.RemoveSession(CallSid);
EnginePool[task].Engine.SignalHangup;
// send a hangup event
end;

end
else // busy, no-answer, canceled, (possible that hangs up before connected, Red button on mobile) etc
begin
EnginePool[task].Engine.SignalMakeCallStatus(CallStatus);
end;

AResponseInfo.ContentText := NoResponse;
AResponseInfo.ContentType := 'text/html;charset="UTF-8"'; // IMPORTANT to have no spaces in ContentType
end;

// Twilio calls this to get MakeCall TwiML, at this point we know MakeCall is connected
if ((PathInfo = '/MakeCall') and (InParams <> '')) then
begin

DefaultLog.WriteTimeStamp(lctSMS, lptNormal, '***** /MakeCall ');
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'Params: ' + InParams + #13#10);

CallSid := aRequestInfo.Params.Values['CallSid'];
engineID := aRequestInfo.Params.Values['engineid']; // value that we sent Twilio via url, so that we know engine#
task := StrToInt(engineID); // TODO: make sure engineid param is there
EngineSessions.AddSession(CallSid, task);
TwiMLResponse := NoResponse;

if not(EnginePool[task].Engine.WaitingOnMakeCallTwiML(TwiMLResponse)) then
begin
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, '***** /MakeCall, Timed out waiting for TwiML, sending Pause ');
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end
else
AResponseInfo.ContentText := TwiMLResponse;

AResponseInfo.ContentType := 'text/html;charset="UTF-8"'; // IMPORTANT to have no spaces in ContentType
end;

// Media Path
if ((PathInfo = '/Media') or (PathInfo = '/MEDIA')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '****Begining MEDIA action: ');
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Params: ' + InParams);
// get file or FILE value
// file=balls.jpg or file=sample.gif, extension determines what ContentType to return
id := ARequestInfo.Params.Values['file'];
if id = '' then id := ARequestInfo.Params.Values['FILE'];

ms := TMemoryStream.Create;

if (id <> '') then
ms.LoadFromFile(id);

ms.Position := 0;

AResponseInfo.ContentType := FileToContentType(id); // based on filename extension
AResponseInfo.ContentStream := ms;

DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'File sent.' + #13#10);

ms.Free; // TODO: try/finally
end; // "/Media"

// email Path
if ((PathInfo = '/Email') or (PathInfo = '/EMAIL')) then
begin
DataModule1.Log.WriteTimeStamp(lctHTTP, lptNormal, '****Begining EMAIL action: ');
DataModule1.Log.WriteTimeStamp(lctHTTP, lptNormal, 'Params: ' + InParams);
DataModule1.Log.WriteTimeStamp(lctHTTP, lptNormal, ARequestInfo.RemoteIP);
DataModule1.Log.WriteTimeStamp(lctHTTP, lptNormal, ARequestInfo.UserAgent);
ip := ARequestInfo.RemoteIP;

SMS := TTwilioSMS.Create(cAccountSID, cTwilioPassword);
sms.Log := DataModule1.Log;

// get file or FILE value
// file=balls.jpg or file=sample.gif, extension determines what ContentType to return
id := ARequestInfo.Params.Values['info'];
if id = '' then id := ARequestInfo.Params.Values['INFO'];
tag := ARequestInfo.Params.Values['tag'];

ms := TMemoryStream.Create;

if (id <> '') then
ms.LoadFromFile('balls.jpg');

ms.Position := 0;

AResponseInfo.CacheControl := 'no-cache'; // don't have browser cache this image
AResponseInfo.ContentType := FileToContentType(id); // based on filename extension
AResponseInfo.ContentStream := ms;

text := tag + #13#10 + ip;
sms.SendSMS('5146295764', '4385001040', text, '', false, numSegments, isMobile, isOptedOut, SMSSid);
sms.Free;

ms.Free;

DataModule1.Log.WriteTimeStamp(lctHTTP, lptNormal, 'Picture sent.' + #13#10);
end;

// Phone Path
if ((PathInfo = '/Phone') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /Phone');
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Params: ' + InParams);

From := aRequestInfo.Params.Values['From'];
privatecall := (From = '+266696687'); // Twilio sends this number when private

From := RemovePlus1(From);
if privatecall then From := 'PRIVATE';

ToDN := RemovePlus1(aRequestInfo.Params.Values['To']);
forwardedFrom := RemovePlus1(aRequestInfo.Params.Values['ForwardedFrom']);
CallSid := aRequestInfo.Params.Values['CallSid'];

FDisplayThread.Display(TimeToStr(Now) + ': Call From: ' + From + ' To: ' + ToDN);
if (forwardedFrom <> '') then FDisplayThread.Display('Forwarded from: ' + forwardedFrom);

DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Incoming Call From: ' + From + ' To: ' + ToDN
+ ' forwarded from: ' + forwardedFrom);

TwiMLResponse := NoResponse; // in case we can't have an engine

// ONLY new calls here
if EnginePool.GetEngineFromPool(task) then
begin
if EngineSessions.AddSession(CallSid, task) then
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Added session: ' + CallSid + ' id: ' + IntToStr(task));
TwiMLResponse := EnginePool[task].IncomingCall(ToDN, From, CallSid, forwardedFrom, privatecall);
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'TwiML(sent back to http server): ' + TwiMLResponse);
AResponseInfo.ContentText := TwiMLResponse;
end
else
begin
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, 'Unable to get an Engine from Pool');
AResponseInfo.ContentText := TTwilioML.NoCapacityCall; // forget this
// AResponseInfo.ContentText := 'Overloaded server';
// AResponseInfo.ResponseNo := 503; // busy, overloaded, not available for now
end;

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';

end; // "/Phone"

// Redirect after a <Say> comes here
if ((PathInfo = '/SayEnd') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /SayEnd');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalSayComplete
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/SayEnd: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// Redirect after a <Play> comes here
if ((PathInfo = '/PlayEnd') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /PlayEnd');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalPlayComplete
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/PlayEnd: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// Redirect after a <Call> comes here (non-action Dial version)
// After coversation ends, busy, no answer or call failed
if ((PathInfo = '/CallEnd') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /CallEnd');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalCallComplete
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/CallEnd: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// <Call> using action, provides DialCallStatus if we need OnNoAnswer, OnBusy
// getting completed means Called Party hung up
if ((PathInfo = '/DialCall') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /DialCall');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

DialCallSid := aRequestInfo.Params.Values['DialCallSid'];
DialCallStatus := aRequestInfo.Params.Values['DialCallStatus'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'DialCallSid: ' + DialCallSid + ' DialCallStatus: ' + DialCallStatus);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalCall(DialCallStatus)
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/DialCall: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// Redirect after a <Gather> comes here
if ((PathInfo = '/GatherTimeout') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /GatherTimeout');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalSpeech('xTimeoutx', '') // we use string "xTimeoutx" to signal user timeout
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/GatherTimeout: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// Redirect after a DTMF comes here
if ((PathInfo = '/DTMFTimeout') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /DTMFTimeout');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalDTMF('xTimeoutx') // we use string "xTimeoutx" to signal user timeout
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/DTMFTimeout: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// Received DTMF, comes here
if ((PathInfo = '/DTMF') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /DTMF');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

DigitsResult := aRequestInfo.Params.Values['Digits'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Digits: ' + DigitsResult);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalDTMF(DigitsResult)
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/DTMF: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// Received speech, <Gather> comes here
if ((PathInfo = '/Gather') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /Gather');
// DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'Params: ' + InParams);

CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);
Confidence := aRequestInfo.Params.Values['Confidence'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'score: ' + Confidence);

SpeechResult := aRequestInfo.Params.Values['speechResult'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'speechResult: ' + SpeechResult);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalSpeech(SpeechResult, Confidence)
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/Gather: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;

end;

// Received speech Partial, <Gather> comes here
if ((PathInfo = '/GatherPartial') and (InParams <> '')) then
begin
CallSid := aRequestInfo.Params.Values['CallSid'];
StableSpeechResult := ARequestInfo.Params.Values['StablespeechResult'];
seqNumber := ARequestInfo.Params.Values['SequenceNumber'];
UnStableSpeechResult := aRequestInfo.Params.Values['UnstableSpeechResult'];

if (cbLogPartialSpeech.IsChecked) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /GatherPartial');
// DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Params: ' + InParams);
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Stable speechResult: ' + StablespeechResult);
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Sequence Number: ' + seqNumber);
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Unstable speechResult: ' + UnstableSpeechResult);
end;

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.maybeSignalSpeechEnd(UnstableSpeechResult, StableSpeechResult)
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/GatherPartial: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.NoResponse;
end;

// action for <Pay> TwiML for <TwilioPay>
if ((PathInfo = '/TwilioPay') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /TwilioPay');
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Params: ' + InParams);
CallSid := aRequestInfo.Params.Values['CallSid'];
TwilioPayResult := aRequestInfo.Params.Values['result'];
TwilioPayError := aRequestInfo.Params.Values['PaymentError'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

//
if Pos('insufficient funds', TwilioPayError) > 0 then
TwilioPayResult := 'not-enough-funds';

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalTwilioPay(TwilioPayResult)
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/TwilioPay: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// Redirect after a <Conference> comes here
if ((PathInfo = '/Conference') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /Conference');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalConferenceEnd
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/Conference: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// Received RecordingStatus,
if ((PathInfo = '/RecordingStatus') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /RecordingStatus');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

RecordingSid := aRequestInfo.Params.Values['RecordingSid'];
RecordingUrl := aRequestInfo.Params.Values['RecordingUrl'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'Recording: ' + RecordingSid + ' ' + RecordingUrl);

//TODO: maybe assign Sid and URL somewhere in Engine for this task, so that
// we may in real-time play it back, store it etc.

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.RecordingComplete(RecordingSid)
else
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '/RecordingStatus: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// url for ModifyCall API (if we are using the url instead of TwiML parameter)
if ((PathInfo = '/ModifyCall') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /ModifyCall');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
TwiMLResponse := EnginePool[task].Engine.GetModifyCallTwiML
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/ModifyCall: unable to find Call Session');

DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'TwiML: ' + TwiMLResponse);

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TwiMLResponse;
end;

// NOTE: for a user to type into a browser
if (PathInfoUser = '/WEB') then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /Web');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := cWebformHTML;
end;

if ((PathInfo = '/webinput') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /webinput');
AResponseInfo.ContentText := 'Unsuccessful. Try again.';

num := aRequestInfo.Params.Values['num'];
address := aRequestInfo.Params.Values['address'];
num := Trim(num); address := Trim(address);
if EngineSessions.LookupSession(num, task) then
begin
EnginePool[task].Engine.HTTPEvent(num,address);
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'HTTPEvent');
AResponseInfo.ContentText := 'Successful. Thank You.';
end;

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
end;

// Twilio's <Enqueue> action="" comes here
if ((PathInfo = '/EnqueueAction') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /EnqueueAction');
CallSid := aRequestInfo.Params.Values['CallSid'];
QueueResult := aRequestInfo.Params.Values['QueueResult'];
QueueTime := aRequestInfo.Params.Values['QueueTime'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid + ' QueueResult: ' + QueueResult);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalEnqueue(QueueResult, QueueTime)
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/EnqueueAction: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

// Twilio's <Dial> ... <Queue url="" comes here
// This is where we send back TwiML to caller waiting before connected
if ((PathInfo = '/QueueAction') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /QueueAction');
DequeingCallSid := aRequestInfo.Params.Values['DequeingCallSid']; // <== this caller
CallSid := aRequestInfo.Params.Values['CallSid']; // <=== caller that is waiting on hold, this caller will connect to
QueueTime := aRequestInfo.Params.Values['QueueTime'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'this CallSid: ' + DequeingCallSid + ' other CallSid: + ' + CallSid);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(DequeingCallSid, task) then
EnginePool[task].Engine.SignalDequeueConnected(QueueTime, CallSid)
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/QueueAction: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.EnqueueStdResponse;
end;

// Redirect for our <Queue>
if ((PathInfo = '/Queue') and (InParams <> '')) then
begin
DefaultLog.LineFeed;
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, '***** Begining webmodule default action: /Queue');
CallSid := aRequestInfo.Params.Values['CallSid'];
DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'CallSid: ' + CallSid);

// lookup session and deliver currentEvent
if EngineSessions.LookupSession(CallSid, task) then
EnginePool[task].Engine.SignalDequeueTimeout
else
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, '/Queue: unable to find Call Session');

AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.PauseResponse;
end;

except on E:Exception do
begin
AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.NoResponse;
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, 'Exception occurred in the HTTP server GetCommand');
end;
end;

end;

Remy Lebeau · August 17, 2021

28 minutes ago, Joe Sansalone said:

The application (HTTP Server) stopped responding to requests again.

In what way? Did you verify that ALL of the server events stopped (OnConnect, OnHeadersAvailable, etc)? Or is it just the OnCommandGet event? Do you have a capture of the requests that don't respond?

28 minutes ago, Joe Sansalone said:

This time, I simply "Stopped" and "Started" the TIdHTTPServer without restarting the application and it worked - application started

responding to requests.

Then it can't be a deadlock issue, since the server is still responsive to shutdown and restart, and thus the socket threads are not blocking. So there has to be an issue in the request handling. But there is WAY too much code provided with no explanation or context, so it is very difficult to diagnose your issue.

Your OnCommandGet handler is pretty massive (also, there are a lot of places where Exit statements are missing that would avoid unnecessary processing). Have you considered breaking up the code and then unit-testing the pieces individually?

28 minutes ago, Joe Sansalone said:

It's just a bunch of "If then begin end" to handle each Pathinfo differently.

And all of it is wrapped in a Try/Except

That MIGHT be part of the problem. You really should not be catching unhandled exceptions, but if you do, make sure to RE-RAISE any Indy exceptions you happen to catch (they all derive from EIdException). The server needs to handle those internally in order to close the socket and stop its owning thread (unless you close the socket manually in your except block).

28 minutes ago, Joe Sansalone said:

Text := IndyTextEncoding_UTF8.GetString(ToBytes(aRequestInfo.Params.Values['Body'], IndyTextEncoding_8Bit));

Why are you doing that? The Params have already been decoded before OnCommandGet is called (assuming TIdHTTPServer.ParseParams is true, which it is by default), and UTF-8 is the default charset used for that decode (unless the request specifies a charset in its Content-Type header). Why can't you use the Body text as-is?

Text := aRequestInfo.Params.Values['Body'];

Since the text was already pre-decoded, re-encoding it and decoding it as a potentially different charset is just asking for trouble. This is a good way to corrupt the text.

If you absolutely need custom decoding, then you should instead parse the ARequestInfo.UnparsedParams, ARequestInfo.QueryParams, and/or ARequestInfo.FormParams properties as needed. That is the raw data where TIdHTTPServer decodes the ARequestInfo.Params data from.

28 minutes ago, Joe Sansalone said:

if EngineSessions.LookupSession(From + toDN, task) then

begin
EnginePool[task].Engine.SMSEvent(From, toDN, Text, NumMedia, numSegments);
DefaultLog.WriteTimeStamp(lctSMS, lptNormal, 'SMSEvent');
end
else // new
if EnginePool.GetEngineFromPool(task) then
begin
EngineSessions.AddSession(From + toDN, task);
EnginePool[task].IncomingSMS(toDN, From, Text, NumMedia, numSegments,
aRequestInfo.Params.Values['MediaUrl0'], // 1st Media URL for now
aRequestInfo.Params.Values['MediaContentType0']);
AResponseInfo.ContentText := NoResponse;
end
else
begin
DefaultLog.WriteTimeStamp(lctSMS, lptImportant, 'Unable to get an Engine from Pool');
AResponseInfo.ContentText := TTwilioML.NoCapacitySMS; // for get this
// AResponseInfo.ContentText := 'Overloaded server';
// AResponseInfo.ResponseNo := 503; // busy, overloaded, not available for now
end;

Are you sure all of your Engine code is thread-safe and non-blocking?

28 minutes ago, Joe Sansalone said:

AResponseInfo.ContentType := 'text/html;charset="UTF-8"'; // IMPORTANT to have no spaces in ContentType

What is that supposed to mean? You can absolutely have a space after the ';' character, that will work just fine:

AResponseInfo.ContentType := 'text/html; charset="UTF-8"';

Alternatively, simply use the separate AResponseInfo.CharSet property instead:

AResponseInfo.ContentType := 'text/html';
AResponseInfo.CharSet := 'UTF-8';

28 minutes ago, Joe Sansalone said:

ms := TMemoryStream.Create;

if (id <> '') then
ms.LoadFromFile(id);

ms.Position := 0;

AResponseInfo.ContentType := FileToContentType(id); // based on filename extension
AResponseInfo.ContentStream := ms;

DefaultLog.WriteTimeStamp(lctHTTP, lptNormal, 'File sent.' + #13#10);

ms.Free; // TODO: try/finally

This is wrong. One, because the LoadFromFile() is a potential exception waiting to happen, since you appear to be loading files using relative paths. ALWAYS use absolute paths instead! But more importantly, you are freeing the TMemoryStream that is assigned to the AResponseInfo.ContentStream property. Indy DOES NOT make a copy of that stream, so you are leaving behind a dangling pointer that will crash when the server tries to write the stream data to the client after the OnCommandGet handler exits. The AResponseInfo object will free the ContentStream for you when it is no longer needed, so DO NOT free it manually. Unless you really need to free the stream manually (which you don't in this cae), in which case you would have to call AResponseInfo.WriteHeader() and AResponseInfo.WriteContent() manually before freeing the stream.

28 minutes ago, Joe Sansalone said:

// email Path
if ((PathInfo = '/Email') or (PathInfo = '/EMAIL')) then

You should consider using Indy's TextIsSame() function for case-insensitive string comparisons.

28 minutes ago, Joe Sansalone said:

ms := TMemoryStream.Create;

if (id <> '') then
ms.LoadFromFile('balls.jpg');

ms.Position := 0;

AResponseInfo.CacheControl := 'no-cache'; // don't have browser cache this image
AResponseInfo.ContentType := FileToContentType(id); // based on filename extension
AResponseInfo.ContentStream := ms;

text := tag + #13#10 + ip;
sms.SendSMS('5146295764', '4385001040', text, '', false, numSegments, isMobile, isOptedOut, SMSSid);
sms.Free;

ms.Free;

DataModule1.Log.WriteTimeStamp(lctHTTP, lptNormal, 'Picture sent.' + #13#10);

Same here. Also, is the SMS and Twilio code thread-safe and non-blocking?

28 minutes ago, Joe Sansalone said:

if (cbLogPartialSpeech.IsChecked) then

This is NOT thread-safe! You CANNOT access UI controls from outside of the context of the main UI thread. The OnCommandGet event is fired in the context of a worker thread, so you MUST synchronize with the main UI thread here. Otherwise, I would suggest storing the CheckBox value in a global Boolean variable at server startup, and then you can read that variable without synchronizing it (as long as you don't alter the vaiable's value while the server is running, otherwise you do need to synchronize it).

28 minutes ago, Joe Sansalone said:

except on E:Exception do

begin
AResponseInfo.ContentType := 'text/html;charset="UTF-8"';
AResponseInfo.ContentText := TTwilioML.NoResponse;
DefaultLog.WriteTimeStamp(lctHTTP, lptImportant, 'Exception occurred in the HTTP server GetCommand');
end;
end;

If your OnCommandGet code encounters an unhandled exception, you are catching it to send a response to the client, but you are not setting an appropriate ResponseCode, like 500. So the default 200 will likely be sent instead. Is that what you really want? By default, TIdHTTPServer already sends a 500 response to the client for unhandled exceptions, so this code is largely unnecessary. But since you are customizing the ContextText, then at the very least I would suggest getting rid of the outer try/except altogether and move this code into the OnCommandError event instead.

Joe Sansalone · August 17, 2021

36 minutes ago, Remy Lebeau said:

In what way? Did you verify that ALL of the server events stopped (OnConnect, OnHeadersAvailable, etc)? Or is it just the OnCommandGet event? Do you have a capture of the requests that don't respond?

Unfortunately, HTTP logging was off .. I'll capture on the next time.

37 minutes ago, Remy Lebeau said:

Your OnCommandGet handler is pretty massive (also, there are a lot of places where Exit statements are missing that would avoid unnecessary processing). Have you considered breaking up the code and then unit-testing the pieces individually?

That MIGHT be part of the problem. You really should not be catching unhandled exceptions, but if you do, make sure to RE-RAISE any Indy exceptions you happen to catch (they all derive from EIdException). The server needs to handle those internally in order to close the socket and stop its owning thread (unless you close the socket manually in your except block).

Ok, I won't catch unhandled exceptions. So far, there has never been a log to indicate any. And logging for this is always on.

47 minutes ago, Remy Lebeau said:
Why can't you use the Body text as-is?
Text := aRequestInfo.Params.Values['Body'];

Twilio sends the data in SMS texts with emoticons, emojis etc. Using aRequestInfo.Params.Values['Body'] doesn't work.

49 minutes ago, Remy Lebeau said:

Are you sure all of your Engine code is thread-safe and non-blocking?

It is thread-safe. However, it does block for some calls with a maximum timeout so it returns for sure.

Is there a timeout where it's too much for TIdHTTPServer in terms of waiting for CommandGet event?

I can change some of the blocking timeouts to be lower.

54 minutes ago, Remy Lebeau said:

But more importantly, you are freeing the TMemoryStream that is assigned to the AResponseInfo.ContentStream property. Indy DOES NOT make a copy of that stream, so you are leaving behind a dangling pointer that will crash when the server tries to write the stream data to the client after the OnCommandGet handler exits. The AResponseInfo object will free the ContentStream for you when it is no longer needed, so DO NOT free it manually.

Oops! I will fix that.

56 minutes ago, Remy Lebeau said:

if (cbLogPartialSpeech.IsChecked) then

This is NOT thread-safe! You CANNOT access UI controls from outside of the context of the main UI thread. The OnCommandGet event is fired in the context of a worker thread, so you MUST synchronize with the main UI thread here. Otherwise, I would suggest storing the CheckBox value in a global Boolean variable at server startup, and then you can read that variable without synchronizing it (as long as you don't alter the vaiable's value while the server is running, otherwise you do need to synchronize it).

I'm aware that UI controls need to be accessed within the main UI thread.

In this specific case, I verified that cbLogPartialSpeech.IsChecked really only reads a boolean (nothing else)

and I need to change it sometimes while the server is running .. so I took the liberty to do it this way.

1 hour ago, Remy Lebeau said:

If your OnCommandGet code encounters an unhandled exception, you are catching it to send a response to the client, but you are not setting an appropriate ResponseCode, like 500. So the default 200 will likely be sent instead. Is that what you really want? By default, TIdHTTPServer already sends a 500 response to the client for unhandled exceptions, so this code is largely unnecessary. But since you are customizing the ContextText, then at the very least I would suggest getting rid of the outer try/except altogether and move this code into the OnCommandError event instead.

In the case of using Twilio, I need to send something to tell Twilio what to do with the "Call" or "SMS" - something that won't abruptly end the call.

Thus, I send something in their XML-like language to tell them not to drop the call etc. Sending a 500 response will evoke their default behavior to thrash the

existing call. We prefer otherwise. But we may change this ... I understand your concern.

Thanks for the help - much appreciated!

Sorry for the massive code. So much happens in the engine that the

HTTP event code was left to grow without possibly making it more readable.

It's the reason I simply made each section an IF then for each PathInfo,

without using else's or it would really be hard to read.

Joe Sansalone · August 17, 2021

I forgot to mention that each section in the CommendGet event has

been tested quite a bit with no known problems.

I'll have more HTTP logs the next time.

Just realized that even though Checkbox.IsChecked simply accesses

a boolean and does nothing else, it's possible that Windows OS may

require that only the UI thread does this?? Although I thought it

was because of thread-safety and avoiding weird collisions of updating the UI.

In any case I have changed that code.

Edited August 17, 2021 by Joe Sansalone
forgot something

Remy Lebeau · August 17, 2021

1 hour ago, Joe Sansalone said:

Twilio sends the data in SMS texts with emoticons, emojis etc. Using aRequestInfo.Params.Values['Body'] doesn't work.

Can you provide an example? And is the data coming from the URL querystring, or the request body? I assume the latter. It makes a difference to how HTTP behaves.

The code you showed using UTF8.GetString() only makes sense if TIdHTTPServer were storing UTF-8 bytes as Chars in a String, which it should not be doing unless the request specifies a charset (in the Content-Type header) which Indy does not recognize/support. Otherwise, it should be using that charset properly, or if there is no charset specified then it uses UTF-8 (assuming you are using an up-to-date version of Indy). So, it is a little hard to imagine that if Twilio is sending UTF-8 data for the 'Body' text that TIdHTTPServer would not be decoding it properly by default. But, that goes back to my earlier comment about needing to see the raw HTTP requests.

1 hour ago, Joe Sansalone said:

Is there a timeout where it's too much for TIdHTTPServer in terms of waiting for CommandGet event?

No. It will happily wait as long as it takes to read in the request data and send out the response data.

Joe Sansalone · August 17, 2021

NOTE: Twilio server receives SMSs sent to a phone number that resides in their network. They then send an HTTP request to my

server indicating that they received an SMS and they put the contents of the SMS in the Body parameter.

Here is an example of receiving an SMS (i.e. text on a cellular phone) Hello followed by 2 different smiley face emojis, received by Twilio and sent

as a request to my server:

Unparsed Param: Body=Hello+%F0%9F%98%85%F0%9F%A4%A3

Parsed Param: Body=Hello ð???ð?¤£

This is perfectly fine.

But, I found that then I need to do this when receiving:

Text := IndyTextEncoding_UTF8.GetString(ToBytes(aRequestInfo.Params.Values['Body'], IndyTextEncoding_8Bit));

in order to SEND the same SMS with the same Hello and 2 smiley faces using the REST components:

FRestRequest.Params.AddItem('Body',TIDURI.ParamsEncode(aText, IndyTextEncoding_UTF8) , TRestRequestParameterKind.pkGETorPOST,
[poDoNotEncode]);

If I code it the regular way, it doesn't work.

It sends the outgoing SMS with Hello .. but the 2 smiley faces are weird characters instead of 2 emojis.

Remy Lebeau · August 18, 2021

On 8/17/2021 at 3:47 PM, Joe Sansalone said:

Unparsed Param: Body=Hello+%F0%9F%98%85%F0%9F%A4%A3

Parsed Param: Body=Hello ð???ð?¤£

It SHOULD NOT be decoding the unparsed param like that. The encoded hex shown is valid UTF-8, and I can decode it manually into the two Emojis (U+1F605 and U+1F923). But the parsed param you have shown is the result of Indy storing the raw UTF-8 bytes into the String rather than decoding the UTF-8 bytes into Unicode text.

Like I said earlier, TIdHTTPServer decodes the params using UTF-8 by default, unless overridden by an explicit charset in the request.

Oh, wait... (checks Indy's change history...) that was actually a recent fix made earlier this year. Prior to that fix, Indy would indeed decode the Param data using 8bit instead of UTF-8, unless the request explicitly stated UTF-8. So that would explain the behavior you are seeing, and why the IndyTextEncoding_UTF8.GetString() workaround was needed.

You should update to the latest version of Indy. Then the param data should be decoded properly as UTF-8 by default and you won't need the IndyTextEncoding_UTF8.GetString() workaround anymore.

Quote

This is perfectly fine.

No, it is not fine. It is broken, actually. But it was fix several months ago.

Quote

But, I found that then I need to do this when receiving

Yes, but you shouldn't have had to resort to that if Indy were decoding the param data correctly to begin with. Which clearly your version is not.

Edited August 19, 2021 by Remy Lebeau

Joe Sansalone · August 18, 2021

Ok got it.

When I update to latest Indy version (either with next Delphi 11 or manually),

I will make sure to change my code.

Thanks!

Joe Sansalone · September 8, 2021

I got the PROBLEM again - HTTP Server stops responding.

This time I logged OnConnect, OnHeadersAvailable, OnException, OnCommandError events as well as

the usual OnCommandGet, OnCommandOther.

There was NO activity in the above events when the problem occurred.

Below is the log from Twilio's server which sends a request to my server (live.projectone.ca) :

========================================================================================

MESSAGE: An attempt to retrieve content from https://live.projectone.ca/Phone returned the HTTP status code 502

SSL/TLS Handshake Error

Error - 11220

SSL/TLS Handshake Error

During SSL/TLS negotiation, Twilio experienced a connection reset.

Possible Causes

Incompatible cipher suites in use by the client and the server. This would require the client to use (or enable) a cipher suite that is supported by the server.

Possible Solutions

Verify cipher suites in-use are up to date. Twilio-supported ciphers can be found here
Use compatible version of TLS, Twilio supports TLS 1.0,1.1,1.2.

============================================================================================

After closing/starting the application again, everything was fine.

Is it possible that there's a newer version of Indy that fixes this problem?

I'm thinking of running without SSL to see if this problem occurs.

Joe

Remy Lebeau · September 8, 2021

6 hours ago, Joe Sansalone said:

I got the PROBLEM again - HTTP Server stops responding.

Did you do what Twilio said to do? Did you verify the TLS version and ciphers being used by your server during a failed connection? Have you tried sniffing the TLS handshake with Wireshark to see what is actually going on inside of it?

Quote

During SSL/TLS negotiation, Twilio experienced a connection reset.

The only time I have ever seen that happen is when OpenSSL decides to reject a handshake and just closes the TCP connection abruptly without first sending a TLS alert to explain why it is being rejected. That usually implies that something in the handshake is corrupted or incorrect, and OpenSSL is just being cautious about sending anything, but that is not always the case.

Quote

Is it possible that there's a newer version of Indy that fixes this problem?

I doubt it, since this is not really an Indy issue, it is more of an OpenSSL TLS issue, but whether it is on the client side or the server side is still TBD.

Edited September 8, 2021 by Remy Lebeau

Joe Sansalone · September 8, 2021

Why does it work for days and then it doesn't?

Strange. It's the same instance. I don't change anything.

I'll install Wireshark on the server.

Thanks for your help.

Remy Lebeau · September 8, 2021

1 hour ago, Joe Sansalone said:

Why does it work for days and then it doesn't?

I can't answer that. There simply has not been enough information provided yet to diagnose the root cause. You are just going to have to continue debugging deeper.

sp0987 · April 17

HTTPServer : Tidhttpserver

with httpserver do begin
HTTPServer.DefaultPort := 8080;
HTTPServer.Bindings.Clear;
HTTPServer.Bindings.Add.port := 8080;

if active then stop else active.

end

procedure HTTPServerCommandGet(AContext: TIdContext; req: TIdHTTPRequestInfo; res: TIdHTTPResponseInfo);

begin

try

res.SERVER := servername;
res.ContentType := 'text/html;charset="UTF-8"';
res.ResponseNo := 200;

log(req.httpcommand); --// If i remove this then am getting EInvalidPointer : Invalid pointer operation
17:01:23.675 - /images/RuleCOMPILEDSCRIPT.png : /images/RuleCOMPILEDSCRIPT.png for all types of files

// if the req.documet is other than .png/.jpg

Data := LoadDataFromFile(doc);

res.ContentType := fMIMEMap.GetFileMIMEType(doc);
res.ContentStream := TStringStream.Create(Data);

// if the req.documet is .png/.jpg

res.ContentType := fMIMEMap.GetFileMIMEType(doc);
res.ContentStream := TFileStream.Create(doc, fmOpenRead or fmShareCompat);

except
on E: Exception do
begin
Log(req.URI + ' : ' + req.Document + sLineBreak + E.ClassName + ' : ' + E.Message);

end;
end;

end

Am getting EInvalidPointer : Invalid pointer operation while processing one get request to another get request. If the log(req.httpcommand) is used then am not getting any invalid pointer operation exception.

Attached sample log file

log - Copy.txt

Remy Lebeau · April 17

6 hours ago, sp0987 said:

with httpserver do begin

HTTPServer.DefaultPort := 8080;
HTTPServer.Bindings.Clear;
HTTPServer.Bindings.Add.port := 8080;

if active then stop else active.

end

There is no reason to assign a value to DefaultPort if you are creating only 1 Binding and setting its Port, thus overwriting what the DefaultPort already assigned. On the other hand, there is no point in assigning the Binding's Port if you are assigning it the same value as DefaultPort. The whole point of DefaultPort is it is the initial Port that is assigned by Add().

6 hours ago, sp0987 said:

log(req.httpcommand); --// If i remove this then am getting EInvalidPointer : Invalid pointer operation

17:01:23.675 - /images/RuleCOMPILEDSCRIPT.png : /images/RuleCOMPILEDSCRIPT.png for all types of files

Such weird behavior is typically a classic symptom of "Undefined Behavior" being invoked elsewhere in your code, and this code is likely just an unexpecting victim of it. For instance, if random memory were being corrupted, and then this code tries to access that memory.

6 hours ago, sp0987 said:

// if the req.documet is other than .png/.jpg

Data := LoadDataFromFile(doc);

res.ContentType := fMIMEMap.GetFileMIMEType(doc);
res.ContentStream := TStringStream.Create(Data);

Why are you using a TStringStream for raw file data? If you really need to load the file data into a String (ie, the file is textual), then you could simply use ContentText instead of ContentStream, no need to use a TStream.

6 hours ago, sp0987 said:

except

on E: Exception do
begin
Log(req.URI + ' : ' + req.Document + sLineBreak + E.ClassName + ' : ' + E.Message);

end;

As I mentioned earlier in this thread, if you catch exception, you should RE-RAISE any Indy exceptions that are caught, eg:

except
  on E: Exception do
  begin
    Log(req.URI + ' : ' + req.Document + sLineBreak + E.ClassName + ' : '  + E.Message);
    if E is EIdException then raise; // <-- ADD THIS
  end;

Sign In

Indy HTTP server

Recommended Posts

Joe Sansalone 6

Share this post

Link to post

Remy Lebeau 1409

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Remy Lebeau 1409

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Remy Lebeau 1409

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Remy Lebeau 1409

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Remy Lebeau 1409

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Remy Lebeau 1409

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Joe Sansalone 6

Error - 11220

SSL/TLS Handshake Error

Possible Causes

Possible Solutions

Share this post

Link to post

Remy Lebeau 1409

Share this post

Link to post

Joe Sansalone 6

Share this post

Link to post

Remy Lebeau 1409

Share this post

Link to post

sp0987 0

Share this post

Link to post

Remy Lebeau 1409

Share this post

Link to post

Create an account or sign in to comment

Create an account

Sign in

Browse

Activity