Updated XMLMapper

[CoeurdeLeon 8]

A month ago Miguel Angel Moreno presented a video on new features in XMLMapper.  Is the new version included in Delphi 11.3?

 

Dick Maley

[Lars Fosdal 1792]

The version resource of the default installed XMLMapper says 28.0.47991.2819 which only indicates it was built at the same time as the IDE.

It still can't handle a recursive .xsd definition file.
 

But, in GetIt - you'll find

 

and that can handle my recursive .xsd - so that version appears to be the one with the new features!

 

[CoeurdeLeon 8]

Thank you Lars

[Lars Fosdal 1792]

Please note that since it still is marked as beta - there may be issues.

If you find something, please report it.

[CoeurdeLeon 8]

Will do

[Miguel Moreno 7]

Lars and CDL:

 

Yes, the final release of XML Mapper 11.3 is just around the corner and for right now it will still be distributed via GetIt.

 

In the mean time, as suggested by Lars, you can get the 11.3 Beta XML Mapper to explore the new features.  The companion document highlights the bugs fixed and the limitations of the beta.

 

I will let you know once the final release is available for you to install.

 

PS:  Yes !!  If you find something please report it via Quality Central !!

[CoeurdeLeon 8]

I have just finished upgrading to 11.3 and the first item I have installed is the new XMLMapper.

 

I CANNOT FIND IT.

 

All I can find is the old one.  Where is it?

 

Miguel, I really liked your recent webinar and look forward to your continued progress on this project.

 

Dick Maley

[Miguel Moreno 7]

Hello Dick.

 

For the new Delphi Alexandria 11.3,  the included XML Mapper is still the old legacy version.

 

To get the "XML Mapper Malawi Beta" version in the mean time, you have to launch the "GetIt Package Manager" from the tools menu ...

Once you get the "GetIt" main dialog, type "xml mapper" in the search box ...

The "XML Mapper Malawi Beta" will show up in the available packages list.  Click on the "Install" button to install it in your Delphi environment ...

[CoeurdeLeon 8]

Miguel

 

I have done just as you said

 

And when I launch XMLMapper from the tools menu I get

 

Is the new XMLMapper found someplace else?

Dick Maley

 

[CoeurdeLeon 8]

I fear the error is mine.

 

The Beta package is 28.0.47741.2595

 

Launching XMLMapper from the tools menu and examining its about window I get

 

The versions match.

I was expecting to see the new filters on the left side.

These are displayed when the schema view is selected.

Miquel and Lars, thank you for your help.

Dick Maley 

[Miguel Moreno 7]

Dick:

 

The filters you expect to see  are there  ...     Just switch to the "Schema View" tab on the left side panel 😊

 

Those filters apply only when you load an XML XSD schema ...

 

When loading a raw XML file the filters do not apply, so that is the reason you do not see them when the "Document View" tab is selected.

 

And do not feel bad ...   We also feel that the source panel ( left side one ) of XML Mapper is a bit confusing ...

 

We would like to overhaul the user interface to make it more intuitive, but for right now our main priority is to make XML Mapper fully usable, bug and crash free, and to support all modern XML documents and schemas.

 

Please report any issue or bug that you find using the "Malawi Beta" XML Mapper ...

[Miguel Moreno 7]

Lars and Dick:

 

I am please to inform you that Embarcadero has released the official "XML Mapper" for Delphi 11.3, internal version 28.0.47991.2819, via the GetIt package manager.

 

We have made an extensive effort to make XML Mapper reliable and usable with any modern XSD and/or XML file.

 

Please give it a try and let us know of any issue you may find through Embarcadero's Quality Central.

 

Thanks.

 

[Lars Fosdal 1792]

Excellent work, @Miguel Moreno!

Is there a way to control the number of levels for recursive XSD types?

[Roger Cigol 103]

Hi Lars, There is a numeric entry box for this limit on the XSD tab

 

[Lars Fosdal 1792]

Perfect. I should have seen that, but I only tested it on a single .xsd file and was just waiting for the crash that didn't happen.

[Miguel Moreno 7]

Lars:

 

Hahahaha 😁

 

Glad to hear that "the crash [...] didn't happen" ...  👌🏼

[Lars Fosdal 1792]

@Miguel Moreno - I have an XSD schema that the XMLMapper doesn't like.

Should I create an entry in Quality Portal?

 

When I try to select the entire structure, I get multiple of these.

 

The document also has more than one root - which seems to be handled.

[Lars Fosdal 1792]

@Miguel Moreno It seems that the XMLMapper the 12.0 GetIt, is the old version again 😞

It crashes on the recursive TPack.xsd, and it fails to correctly render a relatively simple schema, while the newer XMLMapper in 11.3 does the job right.

11.3 version on top, 12.0 version on bottom.

[chgemini 1]

Good morning all, I'm sorry to bump an old post but I could do with some help please.

 

I am using XML Mapper but cannot get the attached XML file mapped, it only has 2 fields but is a soap response, any ideas at all please? I have tried with both the old version and the Malawi version.

 

The files were mapped using XTR files some 10+ years ago by another developer, but the spec has since changed and I cannot see any results when testing the transformation on the file.

 

Thanks so much in advance

 

 

SOAP_GetCAPRangeResponse.txt

Edited March 20 by chgemini

[Roger Cigol 103]

Your file refers to a custom schema with the line

		<GetCAPRangeResponse xmlns="http://www.capnetwork.co.uk/CAPVehicles/">

If you type this URL into your web browser you get a 404 error - ie this schema has been removed or the website is down.

[Miguel Moreno 7]

On 4/20/2023 at 8:04 PM, Lars Fosdal said:

@Miguel Moreno - I have an XSD schema that the XMLMapper doesn't like.

Should I create an entry in Quality Portal?

 

When I try to select the entire structure, I get multiple of these.

 

The document also has more than one root - which seems to be handled.

 

Hello Lars.

 

Sorry for not replying on time...  For some reason my email notifications coming from "Delphi-Praxis" started to go to my "junk" mailbox and I did not noticed until now.  This has been fixed now. 😊

 

Regarding the error message that you are getting (see above):

 

1. We have made big changes in the way that XML mapper handles errors.  In the past, it would simply either ignore all errors -- no error message or feedback to the user -- or not handle them at all -- causing "blind" crashes with no clue or info about why it crashed.
2. As part of the error handling overhauling, we have added several error handlers and we now try to give feedback to the user in all of them, so she/he knows what is going on.
3. One of those errors has to do with a design flaw in the original XML mapper:
   1. XML Mapper was designed to handle "repeating XML elements" (i.e., "nested datasets") based on simple, non-complex elements ( Definition of XML complex elements ).
   2. Originally, XML Mapper "assumed" that the input XML always complied to the rule above, so it didn't even check or validated the XML data -- as a result, XML Mapper would simply hang up when the rule was not met.
   3. Once we detected this case, we added logic to validate the XML data, "catch" the error and inform the end user about the non-compliant issue.

After we made these changes, we have now found several "modern day" XML files and schemas with "complex" XML elements defined as "repeating" ("nested dataset").  As part of our current development milestone -- correctly handle all valid modern day XML files -- we have focused on getting this issue fixed once and for all.

 

We can only share for right now that the task has not been trivial -- it has required us to rewrite considerable parts of the original parsing and XML data structure logic to get it done right.

 

This is currently the key milestone we must complete before releasing our next official XML Mapper version, so you will hear from us very soon when we feel that XML Mapper is now "good to go".

 

In the mean time, I am sorry to inform you that there is nothing you can do about it -- XML Mapper will not be able to process your XML file, but at least you are now been "kindly informed" about it instead of just crashing 😁.

 

Thanks,

 

     Miguel

Edited March 23 by Miguel Moreno

[Miguel Moreno 7]

On 2/5/2024 at 2:57 PM, Lars Fosdal said:

@Miguel Moreno It seems that the XMLMapper the 12.0 GetIt, is the old version again 😞

It crashes on the recursive TPack.xsd, and it fails to correctly render a relatively simple schema, while the newer XMLMapper in 11.3 does the job right.

11.3 version on top, 12.0 version on bottom.

 

Hello Lars.

 

I am sorry, but your screenshots do not make sense...

 

1. Both screenshots show the "XML Mapper" with its new face (VCL Styes) -- only the XML Mapper that we distribute through "GetIt" has the new face lift.
2. One screenshot is for the "Schema view, the other for the "XML View (document)"
3. We have not released any new version of "XML Mapper" for Delphi 12.  What you get in Delphi 12 is the "stock good old XML Mapper".

 

So all of this seems odd.

 

If this is still an issue, can you please elaborate on the actual problem after considering my feedback in the previous paragraphs ?

 

Thanks,

 

     Miguel.

 

( FYI, this is how the "stock XML Mapper" shipped with Delphi looks like: )

Edited March 23 by Miguel Moreno

[Miguel Moreno 7]

On 3/20/2024 at 4:58 PM, chgemini said:

Good morning all, I'm sorry to bump an old post but I could do with some help please.

 

I am using XML Mapper but cannot get the attached XML file mapped, it only has 2 fields but is a soap response, any ideas at all please? I have tried with both the old version and the Malawi version.

 

The files were mapped using XTR files some 10+ years ago by another developer, but the spec has since changed and I cannot see any results when testing the transformation on the file.

 

Thanks so much in advance

 

 

SOAP_GetCAPRangeResponse.txt

 

Hello.

 

Yeah, you are correct, the file you have posted is in fact a "SOAP web service response file".

 

XML Mapper is a tool designed to help you convert ("transform") an XML file that contains data in a repeating pattern to a Delphi dataset data file.

 

To achieve this, XML Mapper expects you to have the XML schema (XSD file) representing the structure of the source XML file you want to convert into a Delphi dataset.  XML Mapper is not designed to read or process web service response files, be it SOAP or REST.  That is a task that you must do separately.

 

Now, let's take a look at your SOAP response file:

 

1. The SOAP response file is correctly formatted:  it has an XML "Envelope" section with header data and an XML "Body" section with the actual payload.

 

2. In the XML "Body" section (payload), we can see first the XMLSchema of the response dataset:

 

3. Right after the XML Schema, we can see the actual XML dataset with its row data:

 

4. We can extract the XML Schema to a separate individual file, adding the same XML header as found in the SOAP envelope.  Let's save it as "SOAP_Dataset_Schema.xsd":

 

5. By the same token, we can extract the XML dataset to a separate individual file, again adding the same XML header as found in the SOAP envelop.  Let's save it as "SOAP_Dataset.xml":

 

6. Open the XML Schema file ("SOAP_Dataset_Schema.xsd") in XML Mapper:

 

7. Create the "transformation" file:

     a) Right click on the schema panel and click on "Select all"

     b) Right click on the schema panel and click on "Create Datapacket from XML"

     c) Click on the center button "Create and Test Transformation"

     d) The test transformation window pops up with sample transformation data

 

8. Load and process the XML dataset (“SOAP_Dataset.xml”):

     a) Click on the “Open file” toolbar icon of the “Test transformation” window

     b) Select and open the “SOAP_Dataset.xml” file

     c) XML Mapper will load and transform the XML dataset

     d) The data will be shown in the “Test transformation” window in a DBGrid with a TClientDataSet as the source

 

I hope this example can guide you on what XML Mapper can do and what you need to do on your own to use it properly.

 

If you have further doubts please let us know.

 

Thanks,

 

     Miguel

 

[chgemini 1]

Excellent, thank you very much Miguel, thats really helpful!