random 404 errors

[Aztec 0]

Hi All.

I am getting random 404 errors when making a call to our indy server.  
When running from postman, I get this

  Quote

 

Error: socket hang up

▶Request Headers

Content-Type: application/json

tenant-url: https://tools.32yo.com/

User-Agent: PostmanRuntime/7.39.0

Accept: */*

Postman-Token: 9d763642-c5d8-42a0-9068-28f38b4185bb

Host: services.32yo.com:8082

Accept-Encoding: gzip, deflate, br

Connection: keep-alive

 

 

Our service is running on Ubuntu 22.04, and is built using the Indy that ships with Delphi 12.1
The service itself is a forking daemon and is created as follows:

 

  Quote

 

 Try
    /// Fork off the parent process
    ProcessID    := fork();
    If (ProcessID < 0) Then
      Raise Exception.create('Error. Cannot fork the process');
    /// If we have a valid PID, then let parent process terminate autonomously.
    If (ProcessID > 0) Then
      Halt(EXIT_SUCCESS);
    /// Change the file mode mask
    umask(0);
    /// Create a new SID for the child process
    SID          := setsid();
    If (SID < 0) Then
      Raise Exception.create('Error in creating a new SID');
    /// Close out the standard file descriptors
    For idx      := sysconf(_SC_OPEN_MAX) Downto 0 Do
      __close(idx);
    /// the Daemon Loop

    ClassesRegistered := false;
    While true Do
      Begin
        StartServer;

        ListenForEvents;

      End;
    AppServer.StopServer;
    ExitCode     := EXIT_SUCCESS;

  Except
    On E: Exception Do
      Begin
      LogMessage('EXCEPTION - LOADER');

        ExitCode := EXIT_SUCCESS;
      End;
  End;

 

 

Indy itself is run as follows:

 

  Quote

 

if assigned(HttpServer) then
    Exit;
  try
    HttpServer                := TIndySparkleHTTPServer.create(nil);
    HttpServer.DefaultPort    := DefaultPort;
    HttpServer.OnCommandOther := DoOnCommand;
    HttpServer.OnCommandError := DoOnError;
    HttpServer.OnParseAuthentication := DoOnParseAuthentication;
    HttpServer.AutoStartSession := true;
    HttpServer.KeepAlive := true;

 

  except
    on E: Exception do
      Log(self, 'EXCEPTION - CreateServer', 'Cant create FHTTPServer :' + E.message);
  end;

 

 

I am probably missing something fundamental, but have no why we get these random failures.

Any help/hints/tips would be most appreciated. If you need more detail of the code, can provide as well.

 

Thanks!!

 

David

 

 

[Remy Lebeau 1554]

  On 5/23/2024 at 1:44 AM, Aztec said:

I am getting random 404 errors when making a call to our indy server. 

The only time that Indy's TIdHTTPServer sends a 404 automatically is if you call AResponseInfo.SmartServeFile() on a file that doesn't exist.  Otherwise, it sends a 404 only if you explicitly set AResponseInfo.ResponseNo to 404 yourself.  So, are you doing either of those in your OnCommandOther event handler?  If not, then are you sure the 404 response is coming from TIdHTPServer, and not maybe from a firewall/router in front of your server that could be intercepting the request and replying to it before TIdHTTPServer sees it?  What do the actual response headers look like?  There is simply not enough info to diagnose your problem.

[Aztec 0]

Hi Remy.

No we are not calling SmartServeFile not are we setting a 404 at all.

We do have Apache setup with a proxy as when we first started the project, Indy didnt have support for the latest ssl

 
 The proxy looks like this

  Quote

  SSLProxyEngine on
    Header set Access-Control-Allow-Origin "*"
    ProxyPreserveHost On
    ProxyPass /empire http://services.32yo.com:8082/empire retry=0
    ProxyPassReverse /empire http://services.32yo.com:8082/empire

 

So we pass a call to an http:// address and then the proxy will send to the https address.
Ideally I would like to not use a proxy, and have the service run on https. I am not sure if we can do this now, with the latest indy? (We run on Ubuntu linux 22.04)

When we get the 404, there are no response headers. It simply gives the error socket hang up as above.

The ports are open in the firewall, and as I say, it is random. Other times we get the response back correctly.

please let me know what other information I can provide and will happily send it over.

We are really at a loss as to explain what is causing this.

 

I am assuming that Indy is happy being set up as a daemon ?

Thanks

 

Cheers

David

 

[Remy Lebeau 1554]

  On 5/23/2024 at 7:27 PM, Aztec said:

We do have Apache setup with a proxy

Did you check the Apache logs for problems?

  On 5/23/2024 at 7:27 PM, Aztec said:

when we first started the project, Indy didnt have support for the latest ssl  ... Ideally I would like to not use a proxy, and have the service run on https. I am not sure if we can do this now, with the latest indy? (We run on Ubuntu linux 22.04)

You can enable SSL/TLS in TIdHTTPServer.  But no, Indy does not natively support an up-to-date version of OpenSSL yet, only up to 1.0.2u, but that does support TLS 1.2.  However, there are other 3rd party options for using newer TLS versions with Indy.  And, nothing stops you from writing a custom TIdSSLIOHandlerSocketBase-derived class to interact with whatever TLS library you want.  So, it is not an ideal situation, but it is not a show-stopper, either.

  On 5/23/2024 at 7:27 PM, Aztec said:

When we get the 404, there are no response headers. It simply gives the error socket hang up as above.

What you showed is not an HTTP 404 error.  It sounds like a communication problem between the proxy and the TIdHTTPServer.

  On 5/23/2024 at 7:27 PM, Aztec said:

The ports are open in the firewall, and as I say, it is random. Other times we get the response back correctly.

Then you are just going to have to keep debugging and logging until you can figure out what reproduces it.

  On 5/23/2024 at 7:27 PM, Aztec said:

I am assuming that Indy is happy being set up as a daemon ?

It should be fine, provided the ports are being blocked by either the firewall or the OS.

Edited May 23, 2024 by Remy Lebeau