AcceptSecurityContext (Negotiate) not working

[stijnsanders 36]

Help! I read that future versions of Windows may put NTLM out with the old unwanted stuff, so, I thought I'd check the alternatives that are proposed. There's AcceptSecurityContext (Negotiate) that looks like it works identically like what I was using before for NTLM. It looks like that works just great on my laptop here...

But at work, it doesn't ofcourse! Negotiate is supposed to automatically switch between Kerberos and NTLM, and at work I do indeed notice other and longer binary buffers, get two SEC_I_CONTINUE_NEEDED responses, but strangely enouch a third time. When the browser (Chrome in this case, I haven't been able to test others, but if they also call InitializeSecurityContext like you're supposed to, there should be no difference) returns the request for the third time, there's no extra data in the Authorization: Negotiate header, en it fails. Even if I try another AcceptSecurityContext, it gives SEC_E_INVALID_HANDLE ofcourse.

I can't seem to find what's wrong. I can't notice any difference to the data in the buffers, or the flags set in pfContextAttr... I've been checking if it's something with AcquireCredentialsHandle, but that one seems all correct... I can't think of anything more I could check for what's going wrong... Should I try to build the same in IIS and have a close look at the data that is passing between IIS and Chrome to compare to my data?

(In case you were willing to debug, the code is here, let me know if you need help setting up a debugging environment as I know the xxm project registry is not self-explanatory.)

[Kas Ob. 121]

Sorry for guessing as i can't debug or test any better than you.

 

10 hours ago, stijnsanders said:

I can't seem to find what's wrong. I can't notice any difference to the data in the buffers, or the flags set in pfContextAttr... I've been checking if it's something with AcquireCredentialsHandle, but that one seems all correct... I can't think of anything more I could check for what's going wrong..

To me that look like enforced policy or missing domain for the user, Windows has strict policy when it comes to domains and users already known in AD.

 

10 hours ago, stijnsanders said:

Should I try to build the same in IIS and have a close look at the data that is passing between IIS and Chrome to compare to my data?

Will definitely give you the reason and the ability to resolve this, but you must duplicate the same configuration for registered domain and their policy ? that is the tricky part, as to my knowledge these are scattered over few services and there is no one resource to explain them all.

 

[stijnsanders 36]

Well, yes with some extensive googling web searching, I learned that the problem is probably something with the domain. I replicated the configuration in IIS, removed "NTLM" and "Negotiate" from Windows Authentication, and only added "Negotiate:Kerberos", and had the same behaviour as I was having: Chrome tries Negotiate, but seems to fail and pops up the username and password dialog.

I guess I'll have to ask the domain admin at work if he can add "http/"+computername. I would like to have seen a windows event log item clearly stating what's wrong, but didn't find any. I guess those get sent to a gigantic log on the domain server somewhere I don't have access to. Or I need to set up a full test-environment here, but not sure if you can on a 'plain' Windows or need a Windows Server instance...

[Kas Ob. 121]

17 hours ago, stijnsanders said:

Well, yes with some extensive googling web searching, I learned that the problem is probably something with the domain. I replicated the configuration in IIS, removed "NTLM" and "Negotiate" from Windows Authentication, and only added "Negotiate:Kerberos", and had the same behaviour as I was having: Chrome tries Negotiate, but seems to fail and pops up the username and password dialog.

Me too !

Going away from Google search, it is becoming useless more and more by the day.

17 hours ago, stijnsanders said:

I guess I'll have to ask the domain admin at work if he can add "http/"+computername. I would like to have seen a windows event log item clearly stating what's wrong, but didn't find any. I guess those get sent to a gigantic log on the domain server somewhere I don't have access to. Or I need to set up a full test-environment here, but not sure if you can on a 'plain' Windows or need a Windows Server instance...

Setting up full test-environment is the highly recommended as i think and also as i remember it is pain in the back, to ask a specific or not very specific questions to someone who have no idea what you are looking for, also here kicks in the terminology to create bigger mess in communications.

 

ps:

I have close enough experience with domain problem, it was the need to remotely access hyper-v manager without domain and without RDP, found few sources and tried them all, all failed, i tried all combination with policies mentioned but no success, then there was a software and i can't remember its name, alas, searching the net also didn't help me recall, the trial version of it did allow me to control and manage the hyper-v services remotely without domain and without any special setup on my PC or on the server, my only regret now that i didn't dig deep into it and its traffic, all what it did need is user and password with server IP.

 

[stijnsanders 36]

Haha, apparently setspn is available in most Windows'es, and on my laptop:

C:\Users\User1>setspn -l User1
FindDomainForAccount: Call to DsGetDcNameWithAccountW failed with return value 0x0000054B
Could not find account User1

that's an oddly specific error! Arent those function calls with two letter prefixes ntdll.dll internal kernel calls?
To get a proper environment set up to test this, I guess I'll have to get a developer license of Windows Server from somewhere, probably in a virtual image...

[Kas Ob. 121]

Question :

 

Have you tried with ASC_REQ_ALLOW_MISSING_BINDINGS ?

And with a combination with SECURITY_NATIVE_DREP or SECURITY_NETWORK_DREP

 

and most perplexing for me this ASC_REQ_DELEGATE 

[Kas Ob. 121]

 

On 11/16/2023 at 8:55 PM, stijnsanders said:

Or I need to set up a full test-environment here, but not sure if you can on a 'plain' Windows or need a Windows Server instance...

I asked a friend to confirm if Hyper-V hosted Windows Server can create a domain and fully control an AD, he indeed confirmed it, also it is possible with not activated server that will serve and run for 6 months, sound great.

 

My problem is in the middle of joining my Windows 10 Pro to that AD, my Windows also the freaking portable Chrome went crazy, and started to ask me to confirm the user password, anyway this is for heads up.

 

Away from that, tried to play with xmm by build it, there is few things to point

1) Doesn't compile on XE8, CompilerVersion in THeapStream is wrong changing it to <30, did the job for XE8.

2) The projectgroup missing few projects.

3) xxmProjects is missing xxmConvertXML and xxmConvert2, not to be found in the repo.

4) What on earth the json file that is needed everywhere and where to get working one, there is no mention to how and what it should contain, i guess xxmProject should built one, so i downloaded the binary package, but can't figure what to put in it as minimum as possible.

[stijnsanders 36]

Thank you for the feedback!!! It's great to finally have someone looking at xxm. I've been trying to find anyone that wants to combine Delphi and web (back-end) in a specifically different way than trying to force RAD-form design onto web-pages (yuck) and also steer clear from ORM's, but that's still something else than the basic project that indeed needs some work just to get it to be more welcoming for new users...

 

1) I have been sticking to Delphi 7 for a very long time, (and trying to keep xxm working for both Delphi 7 and 'modern' Delphi) so I was not sure where exactly the switch to NativeInt was. I'll update the $IF's to <30, I guess.

2) That may be by design, I've got the 'main' projects in the dpg/groupproj, and some periferal projects serve some specific tasks, or are not ready for primetime yet...

3) I'm in the process of removing xxmConvert*, and may have missed a few dpr/dproj, please check my most recent commits

4) You're right, I completely forgot to commit an example "xxm.json". xxmProject should be able to add to an empty file named "xxm.json" when you're using the "Register with xxm.json..." button, I should look into that, at least update the documentation (I had xxm.xml before, but the same problem applied...)

 

For now you should be able to get going with something like this:

 

{
	"projects": {
		"xxm": {
			"path": "D:/Data/xxm/Delphi/demo2/listprojects/listprojects.xxl"
		}
	}
}

Also I would suggest to limit yourself to xxmHttpDev (or just xxmHttp), the other projects are for other web-environments (IIS, SCGI, Apache httpd...)

Edited November 23, 2023 by stijnsanders

[Kas Ob. 121]

14 hours ago, stijnsanders said:

also steer clear from ORM's

I can find words to explain how much i hate ORM, for me it is a pile of hot stuff.. , ORM for me doesn't bring anything useful except unneeded complexity, more code to maintain and break with changes forcing to rebuild the everything and huge dependency on the unknown, uncontrolled and movable parts being data to find them selves as objects in running code.

 

When it comes to HTTP, mostly i use RTC, don't know if you are familiar with it or with its powerful scripting engine, it is just brilliant, the the script it provide bring the PHP syntax ("<? $max(a:10, b:20) ?>") to Pascal, as simple as PHP can generate pages, the RTC scripting generate web pages but with Pascal and Delphi code, here an example

 

But i am always open to know more and find better tooling, and somehow xxm is catching my intrigue, because we can do the same to generate web pages.

So even till now can't run the demos, i see it is close enough syntax to copy and paste, and judging by the design and it might be comparable in speed to RTC and might be lighter on memory than RTC, still missing many features in RTC though, like the right way to handle exceptions, and can't find how xxm handle exception and where are they being raised, although RTC scripting is limited and lacking many important features.

 

1) Web.xxmp is ASCII file and on newer Delphi the xxmproject tries to open it as Unicode so it fail.

2) Still can't manage to run the demos , the paths are not clear for what goes where ?

3) CompilerVersion is in few places, the value is definitely not 30 or should be way more, i suggest to leave but put a comment line in these places to be changed, may be someone here can help in this case

  THeapStream=class(TMemoryStream)
  private
    FHeap:THandle;
  protected
    {$IF CompilerVersion<30}
    function Realloc(var NewCapacity: LongInt): Pointer; override;
    {$ELSE}
    function Realloc(var NewCapacity: NativeInt): Pointer; override;
    {$IFEND}

for me stand alone and IIS module are the same importance.

 

Thank you for your work. 

[Kas Ob. 121]

I suggest/request to add a new download package at http://yoy.be/xxm/download.html , just one binary xxmHttpDev (or xxmHttp) but with full working demos, working out of the box, no interaction needed, something to show the functionality without any setup or tweak.

User or potential users need to see the power of running web server effect in one single click (with xxm features), and the portability of paths and the whole setup, it is important.

[stijnsanders 36]

I'm glad we fully agree on ORM's.

There's something very important you need to know about xxm: it is not a scripting engine.

It uses (the speed and power of) the Delphi compiler to compile a DLL, based on pure-Delphi code it generates from your xxm files that have both HTML and server-side code mixed much like PHP/ASP/ColdFusion script does, but it does not interpret these scripts.

So, to answer your question how exceptions are handled: the compiled DLL is run in the web-environment of your choice, either within xxmHttp.exe that is a HTTP/1.1 server in itself, or by xxmIsapi*.dll that you need to configure properly in IIS. (or the SCGI handler, or the Apache httpd module...)

So if your Delphi logic raises an exception, and it doesn't get handled, it traverses into the 'higher layer', either the default exception handling the xxm handler provides, or worse the system's fallback and the 'default' system exception handling.

Because you build 'real' delphi DLL's with xxm, you have access to anything you would have access to from a 'normal' Delphi project (since, well, it is, it's just auto-generated for you and hidden away in the 'src' folder).

Which is a blessing an a curse, because you're supposed to know what you're doing, take care of security on the server-side, and know that some things are different when running from a DLL inside of a NT-service process... (But don't let that discourage you, most things just work the same.)

 

But your suggestion is a really good one. I'll put a demo package together with xxmHttp.exe and a xxm.json with all the demo projects, to show off how it works. Then, as a first demo you delete some demo*.xxl files, switch to xxmHttpDev.exe and (provided you've got a working DCC32.exe available on your system) see that it really can regenerate the source files and compile the same xxl from source for you...

Edited November 24, 2023 by stijnsanders

[stijnsanders 36]

(Oh one more thing on ORM's: what I do do with databases resulted in this, and if you want to see an example xxm project that uses this extensively, see here)

[Kas Ob. 121]

13 hours ago, stijnsanders said:

There's something very important you need to know about xxm: it is not a scripting engine.

It uses (the speed and power of) the Delphi compiler to compile a DLL, based on pure-Delphi code it generates from your xxm files that have both HTML and server-side code mixed much like PHP/ASP/ColdFusion script does, but it does not interpret these scripts.

I get the jest of that, but i can still serve static files with/without my own scripting , right ?

That why i need to see some action, some clients love to feel like hackers by providing them with some sort of tweaking and adjusting stuff, like building their own SQL query and have the ability to add button for that then adjust the colors of the columns ...etc

13 hours ago, stijnsanders said:

So if your Delphi logic raises an exception, and it doesn't get handled, it traverses into the 'higher layer', either the default exception handling the xxm handler provides, or worse the system's fallback and the 'default' system exception handling.

Because you build 'real' delphi DLL's with xxm, you have access to anything you would have access to from a 'normal' Delphi project (since, well, it is, it's just auto-generated for you and hidden away in the 'src' folder).

Which is a blessing an a curse, because you're supposed to know what you're doing, take care of security on the server-side, and know that some things are different when running from a DLL inside of a NT-service process... (But don't let that discourage you, most things just work the same.)

That literally what i want to hear and read, not to be forced to use others implementation of exception handling and have full control over it.

[stijnsanders 36]

Ok, first thank you again for the suggestion! It was due time that I put some effort in those demo projects again, some were broken since a few releases, and I found a nasty bug that I'll be able to fix by the next release.

So, please head over to https://sourceforge.net/projects/xxm/files/v1.2.7.477/

and check out the file "xxm_demo_with_binaries.zip" (the exe's in them may be 1.2.8 pre-release versions, but have the bug fixed... but that shouldn't matter if you just want to see xxm in action) I've explained how to get things working in the file "readme.html", just extract the folder "xxm_demo" from the zip to somewhere, and everything should work with relative paths.

Edited November 25, 2023 by stijnsanders

[Kas Ob. 121]

Thank you, now things running !

 

One thing though, the xxmProject.exe still refusing to run, it does crash, even when i tried the one from the binaries package it does allowed to select the web file then crash.

From the xxmProject.exe in the Demo package

 

Not really a problem but if the path to the demo (web project) has a space then the compilation fail

 

Tried playing around and i am liking this, one thing though will have time for it later to dig in, the uploading file is slow like it is being throttled, is it ?

[Kas Ob. 121]

And yes compiling on demand is running flawlessly, it is beautiful, short and fast.

 

Congratulations on this piece of gem and thank you !

 

 

ps it is astonishing, the difference between your files which Delphi 7 and the one generated on mine, ten times the size ! (94kb vs 960kb)

[stijnsanders 36]

Oh my yes! How i dream about a 64-bit version of Delphi 7!!! (Are the Lazarus people listening?)

You need to pay attention to details, when trying to keep a project working over Delphi 7 and 'modern' Delphi! Apparently some Explicit* properties bled through from 'modern' Delphi, and Delphi 7 only at run-time doesn't seem to be able to handle them. I'll fix this by xxm's next release.

I've only included xxmProject.exe to demonstrate that there is a UI to edit project configuration, but in theory you can make all edits with notepad to Web.xxmp (it's just JSON actually).

Oh, I'm missing quotes (") in the command line somewhere. I guess you can it's fixed by editing the compileCommand in Web.xxmp to this:

dcc32 "-U[[HandlerPath]]public" -Q "[[ProjectName]].dpr"

The path isn't part of [[ProjectName]], it's [[HandlerPath]] that gets split up, and dcc32 is trying to find the part after the space as if it was a Delphi project...

But I'm glad you got it to work! Have a try making changes to see if you can extend the demo projects with whatever! Demo 04 should work just fine, but 09 is throttled yes, look at line 21

xxm_demo\09 Upload Progress\aAdd.xxm ... (It's to demonstrate the upload progress thing...)

Let me know if you need anything. (Should we start a separate thread on this forum? We're no longer talking about the Negotiate:Kerberos thing anymore...)

[Kas Ob. 121]

12 hours ago, stijnsanders said:

(Should we start a separate thread on this forum? We're no longer talking about the Negotiate:Kerberos thing anymore...)

I don't think it is needed now to separate this thread, and yes we are off the original topic.

 

Still getting to know this gem, now debugging is working, it is somehow big library to get the ins/outs, it is just different and i might think of few additions and adjustments, but all in its time.

 

My question about throttling was due some strange behavior, didn't find yet, but demo09 was throttled and it was visible, demo04 yesterday did something similar but after running many test drives and breaks many things, today i can't repeat it even with the same files, there was some slowness in uploading, demo proto still fail with big files (more than 2.3mb) but not fail with exception just the connection timeout at 30s with my PC and Chrome, also don't think it is huge deal, most important thing is that demos are there and working to tweak, love whole idea of this approach with xxm.

 

Mostly i love the file handling and websocket and the lightness and conciseness of everything around this xxm, it gives more power and control than scripting.

[stijnsanders 36]

Believe it or not, it actually started because I was trying to implement Internet Explorer's IInternetProtocol interface! And I did! For a while it worked and I had URL's working in IE that started with "xxm://"! Because so much was similar to the work I've been doing before on Apache httpd and ISAPI Extensions, I started IxxmContext and concentrated on keeping everything working in whichever environment...