THttpAppSrv processing post despite failed basic auth

omnibrain · February 24, 2022

I'm currently developing an API using THttpAppSrv and custom programmed handlers.

It's intendet to be used as a machine to machine API, so I use basic auth. TLS get's handled by a reverse proxy.

I initialise the Component als follows:

 HttpAppSrv1.AuthTypes:=[atBasic];
 HttpAppSrv1.AddGetHandler('/objects',Tlisaapirada);
 HttpAppSrv1.AddPostHandler('/objects',Tlisaapirada);
 HttpAppSrv1.Start();

My "getPassowrdHandler looks like this. "user" and "pass" are populated during startup of my program.

procedure Tfio_lisaapi.HttpAppSrv1AuthGetPassword(Sender, Client: TObject; var
    Password: string);
var
    ClientCnx  : TMyHttpConnection;
begin
  ClientCnx:=Client as TMyHttpConnection;

  if (ClientCnx.AuthTypes=[atBasic]) and  (ClientCnx.AuthUserName=user) then
        Password := pass;
end;

My AuthResult just does some logging

procedure Tfio_lisaapi.HttpAppSrv1AuthResult(Sender, Client: TObject; Success:
    Boolean);
begin
  if Success then
       Display('Auth: true')
  else
       Display('Auth: false');
end;

When I call my "/objects" endpoint (get and post) with the correct user name and password everything works as expected.

Wenn I use a wrong username or password the "get" gets denied with a "401 Access Denied", as expected.

But for the "post" my "Tlisaapirada.Execute" procedure gets called and executed. This should not happen, instead it should lead to a 401 as well.

Do I miss some configuration on the THttpAppSrv component, or do I need to write special processing?

Angus Robertson · February 24, 2022

If Authentication fails, the client Flags property is set to hg401 so that response is finally returned. The code and sample web servers have checks for h401 in many places, I'd guess your code is checking the flag for GET for not for POST, somewhere. The component does not stop calling events because authentication has failed. Any of those events may also set Flags to hg401 or something else.

Angus

omnibrain · February 24, 2022

Hm, thanks for the answer, but it get's stranger.

I don't have any of the "On*Document" events implemented at all, because all processing happens in my "TLISAAPIRADA.Execute" procedure. I don't serve documents or anything. (I've specified a HttpAppSrv1.DocDir, but that's empty at the moment) The hg401-handling in the examples happens all in the "On*Document" handlers.

Now the funny part is: The first unauthenticated "get" gets a 401 response with

<HTML>

<HEAD>
	<TITLE>401 Access Denied</TITLE>
</HEAD>

<BODY>
	<H1>401 Access Denied</H1>The requested URL /protocol/6192458077624899 requires authorization.<P>
</BODY>

</HTML>

as a body. (other endpoint, same .Execute - refactoring is pending when Auth works...)

The next "get" gets a "501 Unimplemented". Then the next the 401 again and then the next the 501. And so fort...

Where should I put a check if the client Flags are hg401, I I only use my own TUrlHandler class via "Add*Handler"?

I tried adding

   if Flags=hg410 then
     exit;

to the top of my ".Execute", but the compiler complains that hg401 is undeclared (E2003), despite "OverbyteIcsHttpSrv" being in my uses clause.

EDIT: My eyes are getting tired, "hg401" is the correct one. I will try it and get back...

EDIT 2: It looks like "flags" is the one belonging to the URL-Handler and not the one for the client. I can't find how to access the client flags. "client.flags" does not work.

Edited February 24, 2022 by omnibrain

Angus Robertson · February 24, 2022

My main TUrlHandler.Execute functions start with:

if (Client.AuthUserName <> '') then begin

to check for authentication, but the server does the main authentication in the onCheckPassword event which returns false for failure, but does not clear AuthUserName. So there may be a bug for POST. Sorry, don't have time to test it this week.

Angus

omnibrain · February 28, 2022

Checking Client.AuthUserName in .Execute would work for me (if it worked). It would also pave the way for future endpoints with different authorizations per user. I tried it and Client.AuthUserName is always filled, so there might really be a bug as you suspect.

But I still wonder why I even get into .Execute after authenticated failed for the "post", while the "get" produces a "401" (and every other time a 500...) before calling the .Execute.

Angus Robertson · February 28, 2022

I set different levels when checking the password, that determine what data the execute events are allowed to access. I will check the sample works the same with GET and POST in a day or two.

Angus

omnibrain · April 26, 2022

Did you have time to check if there is a bug for POST and why GET produces alternately a 401 and 500 on failed authentication?

Angus Robertson · April 26, 2022

I did some initial testing, and the version of OverbyteIcsSslMultiWebServ in SVN has the comment 'Added authentication using POST requests.' There were no relevant changes to the server itself.

But I'll be doing some more work on POST uploads and authentication next week, so will test it again before the next release.

Angus

omnibrain · March 27

I don't think this works properly in 9.1.

On GET with failed Authentication I get a 401 from ICS with the following document (btw, where can I edit/override this?):

<HTML><HEAD><TITLE>401 Access Denied</TITLE></HEAD><BODY><H1>401 Access Denied</H1>The requested URL /ping requires authorization.<P></BODY></HTML>

On POST I still land in my .Execute-Function. But Client.AuthUserName doesn't get cleared.

I would expect, that like with GET I don"t even come into the .Execute-Function.

The second best solution would be, if the FAuthenticated was available at the Client. So I could check this.

(It took me two years to get back to this, because I was building other endpoints where I do custom jwt processing and had no need for basic auth, in case you wonder)

Angus Robertson · March 27

Basic Authentication has always worked with GET, my own websites have used it for over 15 years, hundreds of logins each day.

If you can reproduce a problem using the multi web server sample, with logs, I'll look into it.

Angus

omnibrain · March 27

6 hours ago, Angus Robertson said:

Basic Authentication has always worked with GET, my own websites have used it for over 15 years, hundreds of logins each day.

GET is working. POST is where I have trouble. I'm going to try to reproduce it with the sample.

omnibrain · March 27

I got a (somewhat working example with minimal changes to the demo.

I used the postinfo-Demo, because I think you added that specifically to test POST.

\demos-data\WebAppServerData\Templates\postinfo.htm is missing, but that doesn't matter for the test.

My one change:

procedure TWeblServerForm.SslHttpAppSrv1AuthGetType(Sender, Client: TObject);
var
    ClientCnx  : TMyHttpConnection;
begin
    //{ It's easyer to do the cast one time. Could use with clause... }
    ClientCnx := TMyHttpConnection(Client);
    if CompareText(ClientCnx.Path, '/DemoBasicAuth.html') = 0 then begin
        ClientCnx.AuthTypes  := [atBasic];
        ClientCnx.AuthRealm := 'DemoBasicAuth';
    end
    else if CompareText(ClientCnx.Path, '/postinfo.html') = 0 then begin // Added for Debugging Basic Auth
        ClientCnx.AuthTypes  := [atBasic];
        ClientCnx.AuthRealm := 'DemoBasicAuth';
    end

GET with no credentials
Request requires authentication "/postinfo.html" AuthType is "atBasic"
192.168.178.105 - WEB-APP /postinfo.htm
Answer Log - 2024-03-27 23:35:45 WEB-APP SYCORAX 0.0.0.0 GET /postinfo.html - 86 - 192.168.178.105 HTTP/1.1 PostmanRuntime/7.37.0 192.168.247.109:86 401 337 218 0
GET with wrong credentials
Request requires authentication "/postinfo.html" AuthType is "atBasic"
AuthGetPassword for "/postinfo.html" AuthType is "atBasic"
[23:37:08 192.168.178.105] authentication result failed with type Basic for /postinfo.html
192.168.178.105 - WEB-APP /postinfo.html
Answer Log - 2024-03-27 23:37:08 WEB-APP SYCORAX 0.0.0.0 GET /postinfo.html - 86 test1 192.168.178.105 HTTP/1.1 PostmanRuntime/7.37.0 192.168.247.109:86 401 337 257 0
GET with correct credentials
Request requires authentication "/postinfo.html" AuthType is "atBasic"
AuthGetPassword for "/postinfo.html" AuthType is "atBasic"
[23:38:01 192.168.178.105] authentication result OK with type Basic for /postinfo.html
192.168.178.105 - WEB-APP /postinfo.html
POST/PUT Content Size 0
Request Content Type

Answer Log - 2024-03-27 23:38:01 WEB-APP SYCORAX 0.0.0.0 GET /postinfo.html - 86 test 192.168.178.105 HTTP/1.1 PostmanRuntime/7.37.0 192.168.247.109:86 200 563 261 0
POST with no credentials

Request requires authentication "/postinfo.html" AuthType is "atBasic"
POST/PUT Content Size 8
Request Content Type: text/plain
Raw Params:
Testdata

Raw Params:

Testdata=

Post URL: http://192.168.247.109:86/postinfo.html
From IP Address: 192.168.178.105
Answer Log - 2024-03-27 23:39:25 WEB-APP SYCORAX 0.0.0.0 POST /postinfo.html - 86 - 192.168.178.105 HTTP/1.1 PostmanRuntime/7.37.0 192.168.247.109:86 200 563 272 0
POST with wrong credentials

Request requires authentication "/postinfo.html" AuthType is "atBasic"
AuthGetPassword for "/postinfo.html" AuthType is "atBasic"
[23:40:33 192.168.178.105] authentication result failed with type Basic for /postinfo.html
POST/PUT Content Size 8
Request Content Type: text/plain
Raw Params:
Testdata

Raw Params:

Testdata=

Post URL: http://192.168.247.109:86/postinfo.html
From IP Address: 192.168.178.105
Answer Log - 2024-03-27 23:40:33 WEB-APP SYCORAX 0.0.0.0 POST /postinfo.html - 86 test1 192.168.178.105 HTTP/1.1 PostmanRuntime/7.37.0 192.168.247.109:86 200 563 311 0
POST with correct credentials

Request requires authentication "/postinfo.html" AuthType is "atBasic"
AuthGetPassword for "/postinfo.html" AuthType is "atBasic"
[23:41:23 192.168.178.105] authentication result OK with type Basic for /postinfo.html
[23:41:23 192.168.178.105] 1: HTTP/1.1 POST /postinfo.html
POST/PUT Content Size 8
Request Content Type: text/plain
Raw Params:
Testdata

Raw Params:

Testdata=

Post URL: http://192.168.247.109:86/postinfo.html
From IP Address: 192.168.178.105
Answer Log - 2024-03-27 23:41:23 WEB-APP SYCORAX 0.0.0.0 POST /postinfo.html - 86 test 192.168.178.105 HTTP/1.1 PostmanRuntime/7.37.0 192.168.247.109:86 200 563 315 0

As you can see, with GET only with correct credentials .execute gets called. With POST .execute always gets called.

Angus Robertson · March 28

Thanks, on my list to investigate, might be a couple of weeks.

Angus

omnibrain · March 28

I debugged the issue for a bit:

In THttpConnection.ProcessPostPutPat Flags gets changed from hg401 to hgAcceptData, so the "case Flags of hg401" never gets triggered.

Perhaps there needs to be a check for the 40x-Flags before the "if (FRequestMethod = httpMethodPost) then TriggerPostDocument(Flags) ..." block.

Quick and dirty, seems to work, but I don't know if something further down the line breaks:

{* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *}
procedure THttpConnection.ProcessPostPutPat;  { V8.08 }
var
    Flags : THttpGetFlag;
begin
    { POST and no content-length received, we treat this as "bad request"     }
    { and don't pass it to the component user.                                }
    if (not FRequestHasContentLength) or (FRequestContentLength < 0) then begin
        { HTTP/1.0
          A valid Content-Length is required on all HTTP/1.0 POST requests.
          An HTTP/1.0 server should respond with a 400 (bad request) message
          if it cannot determine the length of the request message's content.

          HTTP/1.1
          The presence of a message-body in a request is signaled by the
          inclusion of a Content-Length or Transfer-Encoding header field in
          the request's message-headers.
          For compatibility with HTTP/1.0 applications, HTTP/1.1 requests
          containing a message-body MUST include a valid Content-Length header
          field unless the server is known to be HTTP/1.1 compliant. If a
          request contains a message-body and a Content-Length is not given,
          the server SHOULD respond with 400 (bad request) if it cannot
          determine the length of the message, or with 411 (length required)
          if it wishes to insist on receiving a valid Content-Length.

          Currently we act as a HTTP/1.0 server. }
        FKeepAlive := FALSE;
        Answer400;
        { We close the connection non-gracefully otherwise we might receive
          data we cannot handle properly. }
        CloseDelayed;
        Exit;
    end;

{$IFNDEF NO_AUTHENTICATION_SUPPORT}
    if not FAuthenticated then
        Flags := hg401
    else
{$ENDIF}
    if FOutsideFlag and (not (hoAllowOutsideRoot in FOptions)) then
        Flags := hg403
    else
        Flags := hg404;

    case Flags of            // quick and dirty
    hg400:
        begin
            FKeepAlive := FALSE;
            Answer400;
            CloseDelayed;
            exit;
        end;                                                              {/V7.30 }
    hg401:
        begin
            if FKeepAlive = FALSE then {Bjornar}
                PrepareGraceFullShutDown;
            Answer401;
            exit;
        end;
    hg403:
        begin
            if FKeepAlive = FALSE then {Bjornar}
                PrepareGraceFullShutDown;
            Answer403;
            exit;
        end;
    hg404:
        begin
            if FKeepAlive = FALSE then {Bjornar}
                PrepareGraceFullShutDown;
            Answer404;
            exit;
        end;
    end;

    FAcceptPostedData := FALSE;
    if (FRequestMethod = httpMethodPost) then
        TriggerPostDocument(Flags)
    else if (FRequestMethod = httpMethodPut) then
        TriggerPutDocument(Flags)
    else if (FRequestMethod = httpMethodPatch) then
        TriggerPatchDocument(Flags);

    if (not FAcceptPostedData) and (FRequestContentLength > 0) then begin { V7.30 }
    { The component user doesn't handle posted data.               }
    { Turn LineMode off if RequestContentLength > 0, we'll turn it }
    { back on again in our overridden method Receive.              }
        LineMode     := FALSE;
        FPostCounter := FRequestContentLength;
    end
    else
        FPostCounter := 0;

    case Flags of
    hg400:
        begin
            FKeepAlive := FALSE;
            Answer400;
            CloseDelayed;
        end;                                                              {/V7.30 }
    hg401:
        begin
            if FKeepAlive = FALSE then {Bjornar}
                PrepareGraceFullShutDown;
            Answer401;
        end;
    hg403:
        begin
            if FKeepAlive = FALSE then {Bjornar}
                PrepareGraceFullShutDown;
            Answer403;
        end;
    hg404:
        begin
            if FKeepAlive = FALSE then {Bjornar}
                PrepareGraceFullShutDown;
            Answer404;
        end;
    hgAcceptData:
        FAcceptPostedData := TRUE;
    else
        if FKeepAlive = FALSE then {Bjornar}
            CloseDelayed;
    end;
end;

Edited March 28 by omnibrain

Angus Robertson · March 28

Sorry, not looking at complex ICS bugs for a while, catching up on other stuff.

Angus

omnibrain · March 28

11 minutes ago, Angus Robertson said:

Sorry, not looking at complex ICS bugs for a while, catching up on other stuff.

No problem. I just wanted to try it now and report my findings so you have all the information when you find time to look into it.

omnibrain · April 4

Just to add: the

 // quick and dirty

should not handle the 404 case, otherwise nothing works anymore, because the flags comes with hg404 if you "try" static serving first.

Angus Robertson · April 11

On 3/27/2024 at 10:42 PM, omnibrain said:

I used the postinfo-Demo, because I think you added that specifically to test POST.

\demos-data\WebAppServerData\Templates\postinfo.htm is missing, but that doesn't matter for the test.

Took some detective work to work out where that template has gone, it was written last autumn to test a major rejig of web server get/post parameter processing using streams for multi-gig uploads.

But the template never got added to SVN and then got lost when the samples were re-organised.

So I've had to recreate it from my public web site: https://www.telecom-tariffs.co.uk/testing/postinfo.htm

Not in SVN yet, still looking at your real problem.

Angus

Angus Robertson · April 16

I believe the authentication POST problem was mainly a simple literal,

    if FOutsideFlag and (not (hoAllowOutsideRoot in FOptions)) then
        Flags := hg403
    else
        Flags := hg404;

where hg404 should be hgSendDoc. But something else is going on I'm still tracking, RequestDone should be called for a 401 error to reset the state machine, but is not, although it still seems to work.

The biggest problem is our samples test all the authentication variations for virtual and normal pages, but not for template pages or POST pages, so that all had to be added first.

I always test server fixes on my public servers for a day or two, so the changes won't be in SVN until later in the week.

Angus

Angus Robertson · May 17

Authentication should now be fixed in SVN and the overnight zip.

Angus

Sign In

THttpAppSrv processing post despite failed basic auth

Recommended Posts

omnibrain 15

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

omnibrain 15

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

omnibrain 15

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

omnibrain 15

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

omnibrain 15

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

omnibrain 15

Share this post

Link to post

omnibrain 15

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

omnibrain 15

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

omnibrain 15

Share this post

Link to post

omnibrain 15

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

Angus Robertson 574

Share this post

Link to post

Create an account or sign in to comment

Create an account

Sign in

Browse

Activity