An XML DOM with just 8 bytes per node

[Erik@Grijjy 123]

Check out some of the interesting algorithms and data structures used to create an extremely light-weight XML DOM.

 

 

https://blog.grijjy.com/2020/10/07/an-xml-dom-with-just-8-bytes-per-node/

 

[Rollo62 536]

@Erik@Grijjy

Great stuff, thanks for that.

 

What is your next project, are you going to implement similar structure to the JSON parser ?

[Guest]

Great article and neat library, thank you !

Now, i wonder if a memory manager is next on your workshop desk ?
I genuinely think it is a matter of when, not if.

[Erik@Grijjy 123]

49 minutes ago, Rollo62 said:

@Erik@Grijjy

Great stuff, thanks for that.

 

What is your next project, are you going to implement similar structure to the JSON parser ?

Glad you like it! Could be interesting to see if this can by applied to JSON as well. Wouldn't want to break the existing API though...

[Erik@Grijjy 123]

43 minutes ago, Kas Ob. said:

Great article and neat library, thank you !

Now, i wonder if a memory manager is next on your workshop desk ?
I genuinely think it is a matter of when, not if.

Happy to hear! Don't know about creating a memory manager though. Very advanced stuff. And there are already some pretty good 3rd party memory managers out there.

Although it would be nice to have a high performance memory manager for mobile platforms as well. Maybe something like Microsoft's Mimalloc.

[Kryvich 165]

Very interesting library! I am used to use SimpleXML, but if your library is really that fast, I will definitely switch to it. SimpleXML supports ANSI encoding to reduce memory usage, but I think it will not be too difficult to add ANSI to Neslib.Xml.

 

Converting one of my program from SimpleXML to Neslib.Xml was not difficult, but I ran into an exception EIntOverflow in the procedure TXmlPointerMap.Map. Call stack is:

Quote

Neslib.Xml.Types.TXmlPointerMap.Map($406FDB,4765096,$19FDF8)
Neslib.Xml.Types.TXmlPointerMap.Map($B50FF0,0,$B51008)
Neslib.Xml.TXmlAttribute.SetNext(($B51008))
Neslib.Xml.TXmlNode.InternalAddAttribute(5,'4')
Neslib.Xml.TXmlNode.AddAttribute('check2','4')
ExportToXml.DoExportToXml($27E1AF0,TXmlDocument($27D3604) as IXmlDocument)
ExportToXml.ExportToXml

I was able to make a truncated test project to demonstrate the exception, please look at the attachment.

 

P.S. And thank you for the insightful article. I like this type of optimizations!

 

Test Export To XML.zip

Edited October 7, 2020 by Kryvich

[Erik@Grijjy 123]

8 minutes ago, Kryvich said:

Converting one of my program from SimpleXML to Neslib.Xml was not difficult, but I ran into an exception EIntOverflow in the procedure TXmlPointerMap.Map.

Glad you like it, and thanks for pointing out this issue. It only happens when you compile with Overflow Checking turned on. The hash functions I use should be compiled with overflow checking turned off, so I just added an {$OVERFLOWCHECKS OFF} directive to the Neslib.Xml.Types unit. Could you pull the latest version and try again?

[Kryvich 165]

Yes, I can now export my data to XML. The resulting XML looks similar to what I get with SimpleXML. I will test it further.

Thank you!

 

[David Heffernan 2345]

2 hours ago, Erik@Grijjy said:

Glad you like it, and thanks for pointing out this issue. It only happens when you compile with Overflow Checking turned on. The hash functions I use should be compiled with overflow checking turned off, so I just added an {$OVERFLOWCHECKS OFF} directive to the Neslib.Xml.Types unit. Could you pull the latest version and try again?

Would be nice to localise the disabling of overflow checks to just the hashing routines. 

[Erik@Grijjy 123]

6 minutes ago, David Heffernan said:

Would be nice to localise the disabling of overflow checks to just the hashing routines. 

Your wish is my command. Guess I was a bit lazy earlier...

Pushed an updated for this.

[Attila Kovacs 629]

I just felt like with the best cow of the dairy-farm which gives 40 liters of milk then knocks over the bucket.

Can't compile the code because of the inline variables 😕

[Erik@Grijjy 123]

1 minute ago, Attila Kovacs said:

Can't compile the code because of the inline variables 😕

Then maybe now is a good time to upgrade to the latest Delphi version😉

[Alexander Elagin 143]

7 hours ago, Erik@Grijjy said:

Then maybe now is a good time to upgrade to the latest Delphi version😉

No, it is not.

[balabuev 102]

The price is - you cannot have DOMs, smaller than the OS memory page (4KB).

[Erik@Grijjy 123]

7 hours ago, balabuev said:

The price is - you cannot have DOMs, smaller than the OS memory page (4KB).

True, but for such small DOMs, memory isn't the issue anyway, so the overhead is negligible. Although you still get the advantage of reduced memory fragmentation though...

[balabuev 102]

Just as an idea:

FFreeList can be implemented as a linked list, without any additional allocations.