swagger help needed

[David Schwartz 430]

I'm trying to build a small test bed in Delphi for accessing an internal company API and there's a swagger link published for it. What I want to do is suck in the Swagger definition and use it to build something that lets me query the API in Delphi. I'm not sure what all that requires. I guess most of the time people use some kind of javascript framework for this; I want to use Delphi.

 

I searched around on Google and found this project: https://github.com/marcelojaloto/SwagDoc

 

There's a demo in it named 'GenerateUnitFileForMVCFramework' that lets me put in the Swagger URL and it generates a Delphi unit.

 

It works fine, but there may be a bug in this code in that it duplicates every class and interface definition. I was able to fix that without much trouble in the code generator, but there may be a problem in the part that's parsing the Swagger. (Right now I don't care.)

 

I guess the MVC Framework comes from here: https://github.com/danieleteti/delphimvcframework

 

I'm not exactly sure what to do from here. I need a way to connect to the service with OAuth2, and then make some queries and display the results in a form, like in a TListview.

 

I'm looking for suggestions on how to proceed.

 

Thanks!

 

FWIW, someone here wrote a similar thing that generates C# code from a Swagger URL, and that code looks very similar to the Delphi code that's generated by this SwagDoc tool.

 

Edited February 4, 2020 by David Schwartz

[David Schwartz 430]

So I've been messing with this stuff all week, and Swagger / OpenAPI seems to be a way of documenting an API, like a DDL for databases.

 

You can build a UI from a Swagger definition fairly easily, but if you need any kind of authentication, it seems problematic. It's ok for basic testing, tho, especially if no auth is needed.

 

Given the derth of "Swagger to ________" code generators, I'm guessing that's not somethign done very frequently. The main one at swagger.io supports a couple dozen languages, but Delphi and Pascal aren't among them. 

 

SwagDoc is the only one I've found so far, and there's only one demo for generating client-side code. It's got a few bugs, and there's a lot more it could do. The basic Swagger test UI that can be generated is fairly nice for an auto-generated script. You can read and display data very easily. It would be nice if the Delphi code that's generated had sufficient code generated to produce a similar result, but I guess that's left as an exercise for the reader.

 

Has anybody here ever approached things from this direction -- using Swagger / OpenAPI to help generate Delphi client-side code that makes REST calls? I'm curious what you may have found and problems you encountered.

 

I know Delphi has tools to call JSON-REST APIs fairly easily. But when there are nearly 100 APIs and many of the objects the APIs return have nested objects, lists, and/or arrays embedded in them, that complicates things quite a bit.

 

Or maybe the guys who defined this API have gone a bit overboard? It's a long way from most relatively small API calls and what this particular API I'm working with includes.

Edited February 10, 2020 by David Schwartz

[mvanrijnen 123]

I'v created my own swagger (json) to delphi creator.

It creates some model classes, and a base api client, all the models from swagger can be placed in one file, or every class in a separate file.
Currently only working with Swagger v1 json.


I try to put a version online when i get it working with the v3 specs.

I (we) have written a base json client, which can do several methods of authentication, on top on that, i created a base SwaggerClient, and on top of that you get the generated base client, with al the base api calls following the swagger defintion.

example of calling the SWGParser and DelphiGenerator
(its not 100% ready :), the parses should return an swaggermodel, which would be given to the generator for example).

If you have a swagger.json (v1) then i could try to generate the files for you.

 

procedure TfrmSWGTest.CreateFromSwaggerButton9Click(Sender: TObject);
var
  parser : TSWGParser;
  generator : TSWGDelphiGenerator;
begin
  parser := TSWGParser.Create;
  try
    parser.Parse('C:\MySources\MyCompany\Applicaties\MyApplication\Source\API\Source\MySwagger_File.json');
    generator := TSWGDelphiGenerator.Create;
    try
      generator.APIName := 'MyApplicationAPI';
      generator.FilePrefix := 'MyCompany.';
      generator.TypePrefix := 'MyApplicationAPI';
      generator.ModulSubName := 'DTO';
      generator.HTTPMethodPrefix := '/monitoring';

      generator.ExcludeHTTPMethodName := True;
      generator.IncludeCodeComments := False;

      generator.SingleDTOFile := True;
      generator.Generate(parser, 'C:\MySources\MyCompany\Applicaties\MyApplication\Source\API\Generated');

    finally
      generator.Free;
    end;
  finally
    parser.Free;
  end;
end;

Edited February 19, 2020 by mvanrijnen

[David Schwartz 430]

Thanks, but we're using Swagger V2.0.

 

There are C# tools that do this. One thing that makes that really nice is that C# has some constructs that make the classes far smaller than Delphi for the same stuff. Plus, C# supports "partial" class definitions. I don't know why people are so damned allergic to adding stuff like this to the Delphi language. And they want to know why people think Delphi is regarded as "ancient".... sheesh ... it handles a few common use cases really well, and everything else very poorly.

 

That SwagDoc project has the GenerateUnitFileForMVCFramework that I've been playing with. The author did a great job overall separating different parts of the code, but he didn't really separate the Swagger parser from the code generator. (You call the Generate function and it parses the swagger code.) I've been messing with that and discovered they're independent, and it's possible to work with the parser's output for multi-pass code-gen without having to re-parse anything. So perhaps you could use it to upgrade to Swagger V2.

 

Also, I find that although this code appears to be part of an MVC Framework, it's mainly using parts of Teti's DelphiMVCFramework to leverage code for handling Attribute tags on everything. None of the code being generated is specific to MVC per se, as this approach seems to primarily focus on the various Models the API exposes. The V and C parts are left for the user to fill-in.

 

FWIW, it would be far easier to generate C# code from the Swagger data than Delphi code, b/c there's just far less needed for the same goals.

 

That said, there are several different directions one can go with the generated code. The obvious one is a bunch of data classes and a big class that contains methods that implement the API methods. But in looking at our code, I see there are actually two levels of logic: one is a high-level set of methods that take data conforming to the data classes defined in the Swagger spec and pass them to the API methods. The lower-level one is querying the DB and pulling out data and saving it into lists or arrays of data classes from the Swagger spec. Some people here are implementing that lower-level part using an ORM (Entity Framework Core) and some are just doing direct SQL calls. 

 

So there's an opportunity to generate more code that would simplify reaching different goals. Just something to think about.

 

Edited February 19, 2020 by David Schwartz

[mvanrijnen 123]

I also for fun run the C# generator, on the swagger def i'm using, and the generated code is similar to that i generate in Delphi.

You always will have:

- your "data classes", with lists (arrays in my case).

- A big class which implements all the various methods (gets, puts, deletes etc, this will be what your calling from your own client application

- A base client class which does the "lowlevel" http /json etc stuff.

 

C# is not that different from Delphi, i think they are very similar, they have the same designer "Anders Hejlsberg".
It's only that the base code and libs of c# is far more extended then delphi ever will be now days.
 

The partial classes in C# , mainly (as far a i have seen), is handy for seperating forms and there code.

[mvanrijnen 123]

i mean v2 instead v1 sorry for that.

I just see that the swagger def i use, has went to v3, work to do 🙂

 

[David Schwartz 430]

13 hours ago, mvanrijnen said:

C# is not that different from Delphi, i think they are very similar, they have the same designer "Anders Hejlsberg".
It's only that the base code and libs of c# is far more extended then delphi ever will be now days.

I don't know for certain, but I seem to recall there was a language being floated around the time Anders was stolen from Borland that may have turned into C# a year or so later.

 

MS thought C++ had too much irrelevant stuff in it and not enough things for supporting the evolving technologies they were looking at. They had a couple of people on the C++ standards committee who made several proposals that routinely got shot down. 

13 hours ago, mvanrijnen said:

The partial classes in C# , mainly (as far a i have seen), is handy for seperating forms and there code.

"partial" classes are ideal for supporting auto-generated code. They allow you to spread the contents of a given namespace across multiple files. 

 

Personally, I think the whole notion of a "file" as some kind of a meaningful unit of programming has far outlived its useful life. C# has incorporated this in its design, although the programming environment doesn't take much advantage of it. Delphi hasn't even started to move in that direction.

 

The motivation behind this is simple: the IDE should allow us to compose fragments of programs that are saved to a database rather than files, and compose them into larger chunks. Not in contiguous globs of text that we currently call "units" but as little chunks of logic that get glued together as-needed for whatever problem we're solving. 

 

 

Edited February 20, 2020 by David Schwartz

[Monte Carver 0]

David,

 

I happened across your thread, as I have your same need  Did you ever find a solution to parsing a swagger.json into a working delphi unit.  If so can you share knowledge please?

 

Monte

[David Schwartz 430]

On 1/5/2021 at 10:10 AM, Monte Carver said:

David,

 

I happened across your thread, as I have your same need  Did you ever find a solution to parsing a swagger.json into a working delphi unit.  If so can you share knowledge please?

 

Monte

Nope, this died as I got redirected to work on other things. I am still rather surprised that so few tools do it.

[David Heffernan 2353]

On 2/20/2020 at 9:16 PM, David Schwartz said:

Anders was stolen from Borland

I'm pretty sure that he chose to go to MS. And it looks like an inspired decision now. Look what he has built there. 

[David Schwartz 430]

6 hours ago, David Heffernan said:

I'm pretty sure that he chose to go to MS. And it looks like an inspired decision now. Look what he has built there. 

I'm not suggesting he didn't "choose".

 

Average annual salaries were around $50k at the time.

He was reportedly offered $1M to leave Borland.

 

I seriously doubt he would have left for a 10% increase in salary, whatever he was being paid.

 

How many people can you think of who would turn down such an offer?

Edited January 7, 2021 by David Schwartz

[David Heffernan 2353]

2 hours ago, David Schwartz said:

I'm not suggesting he didn't "choose".

You said "stolen" which seemed odd. I don't know where you get your monetary figures from. 

[mvanrijnen 123]

On 1/5/2021 at 6:10 PM, Monte Carver said:

David,

 

I happened across your thread, as I have your same need  Did you ever find a solution to parsing a swagger.json into a working delphi unit.  If so can you share knowledge please?

 

Monte

Using our own swagger parser > delphi unit generator here. works great.  

Generates the needed types as classes and a class which executes the methods. 

 

Will see if i can extract/publish a public version in my spare free time 🙂

 

Edited January 7, 2021 by mvanrijnen

[Lars Fosdal 1793]

1 minute ago, mvanrijnen said:

Using our own swagger parser > delphi unit generator here. works great.  

I am sure many people would love to see that on GitHub.  Any chance?

[mvanrijnen 123]

5 minutes ago, Lars Fosdal said:

I am sure many people would love to see that on GitHub.  Any chance?

Yes, but can't publish the version we are using here (at work). 
I will start soon, creating a new more general version at home (knowledge is in my mind already 🙂 ).

 
 

 

[Monte Carver 0]

That would be... TOTALLY appreciated.  I got started using the delphimvcframework as a template. but there is a lot to it.

[David Schwartz 430]

On 1/7/2021 at 3:01 AM, mvanrijnen said:

Yes, but can't publish the version we are using here (at work). 
I will start soon, creating a new more general version at home (knowledge is in my mind already 🙂 ).

 

Any update on this? I can really use it right now.

 

BTW, I'm still rather astonished nobody has published anything that does this for Delphi -- unless someone did but it's being kept a secret.

 

There are at least a half-dozen for C# and one or more for nearly every language imaginable ... except Delphi.

 

What do people using Delphi do to connect with REST servers that have Swagger specs available? Write their REST interfaces all by hand???

Edited February 28, 2021 by David Schwartz

[mvanrijnen 123]

17 hours ago, David Schwartz said:

Any update on this? I can really use it right now.

 

BTW, I'm still rather astonished nobody has published anything that does this for Delphi -- unless someone did but it's being kept a secret.

 

There are at least a half-dozen for C# and one or more for nearly every language imaginable ... except Delphi.

 

What do people using Delphi do to connect with REST servers that have Swagger specs available? Write their REST interfaces all by hand???

No sorry, did not had time to startup the project (except creating an empty github repos). 

I think i will be starting this month with it.

 

[sh17 26]

I'm in the same situation right now. There's no Delphi client generator for Swagger? Damn it.

[mvanrijnen 123]

Starting soon, but have todo a total rewrite, with the knowledge of the client at work in mind.

Ideas are already there, there will be an swaggger (json) import, a standard baseclient to work with, and a possibility to generate against your own baseclients (self made or emb's).

Will be working with some kind of template then. 

 

Already made a github repo for this,  we can fill ideas overthere or here for starters first till the project is at a minimum usable level.

 

I found a simular C# project which does this, maybe i fork that so i only have to create the delphi generator.

 

 

 

 

Edited April 1, 2021 by mvanrijnen

[mvanrijnen 123]

Okay, started with uploading some old and new work to github.

 

But after some research (followup from a tip from @sh17), we can go a few directions.

 

1. Create a generator in .java for Delphi.  
    (we can fork the code and mustache templates for example from the c# generator and create delphi versions from them

     As the main work is done in the .java code, everyone can "easily" use the mustache templates to create a client against their own base rest-client  )
   Never done jave before 😞

 

 

2.  Continue with what i started now, implement the OpenApi (formeley Swagger) model in Delphi.

     This then could be used to generate the client from native delphi code. 

 

3. 🙂  Generate a generator in .java which generates some kind of inbetween code, usable for us Delphi developers. 
   

 

so we have WiP. (work in progress)

 

The link to the C# Generator:

Part1: https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/AbstractCSharpCodegen.java

Part2: https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/CSharpClientCodegen.java

 

The link to the C# mustache templates:

openapi-generator/modules/openapi-generator/src/main/resources/csharp at master · OpenAPITools/openapi-generator (github.com)

 

Edited April 16, 2021 by mvanrijnen

[David Schwartz 430]

I don't even use Java, so this isn't much help for me.

 

This thing from my original post: https://github.com/marcelojaloto/SwagDoc

and related updates already gets us into the ballpark.

 

The main problem is that it assumes the use of the MVC framework here: https://github.com/danieleteti/delphimvcframework

 

Instead of calling the mvc framework, it needs callbacks or another way to handle data as its parsed.

 

I guess JSON can be a little ambiguous in some cases, which makes it hard to parse?

 

 

[mvanrijnen 123]

 

Yes, ofcourse, can take a look at that, make a fork of that project.

And create a generator which uses the same kind of mustache files as i mentioned.

Project is only OpenApi v2.x, so also need to extend to v3.x 

 

Advantage is that then we are half way to also create the .java later.

I see the .java solution in the longway as the best, because then it can be incorperated in the whole openapi toolchain.

 

But for now, maybe a first expirment to create a mustache based generator on the project mentioned is the best to start.

First client would be created with the standard rest-request component. (or maybe just straight indy http request).

 

 

@David Schwartz Is the API you are using a public one? maybe you have an uri to it ?

 

 

 

Edited April 17, 2021 by mvanrijnen

[David Schwartz 430]

The API I was originally working with is internal, but I've tried reading a number of ones I've found around the internet. There are tons of them.

 

I'm not understanding why java is needed to build something to consume JSON text, build some internal tables, then spit out Delphi classes.

 

From playing with the earlier referenced code, the trickiest part lies in two areas:

 

1) recognizing isomorphic structures (ie, classes with the exact same data members) and combining them into a single definition; and 

 

2) if you allow selecting only certain APIs or groups, then optionally being able to emit only dependent variable and class definitions and skip everything else.

 

The first step is needed to avoid having a bunch of identical classes defined just because the Swagger code defined them and didn't name them, but the parser gave them all unique default names; or they were named in the Swagger text and all were given different unique names for no particular reason.

 

(I saw a situation recently where a common error return structure generated over 50 identical classes where each one had been given a different name. It's completely unnecessary, and simply requires that the structure of these be recognized as identical and a common base class generated that's used for all references. Worst-case, it would be used as the base class for child classes derived from it. I'd lean towards leaving them all referring to the common base class, then maybe add derived classes to distinguise different error returns, or whatever makes sense in the context. A flag to give you a choice might be a nice option.)

 

There's also the matter of how the emitted classes are ordered. The original code sorted them alphabetically, which might not work for Delphi. If you remove the sorting, then they are emitted in the order they're encountered in the Swagger spec, which may not translate into proper ordering for Delphi either. So internally, it needs to have a way of flagging dependencies and using that to emit code in "layers", or at least in an order where dependent types, vars and classes get defined before they're referenced.

Edited April 17, 2021 by David Schwartz

[mvanrijnen 123]

On 4/18/2021 at 1:09 AM, David Schwartz said:

 

 

Its not that difficult, i have my own generator at the moment, only it generates code against our own restclient.  
But it's written just to make  usable classes and client for us.

And only implements at the moment whats needed for us, so i'm trying to determine the specs which the "public" would need, when creating a new one.
 

If i see your comments, your still on swagger 2.x, and we are talking OpenApi 3.x. 

 

The .java would be needed if we (i) want to have a generator which fits the openapi toolchain.

 

Generators List (openapi-generator.tech) 

Create a New Generator (openapi-generator.tech)

 

But i'll see where i land, i go make what i think is necessary, inspect if SwagDog is a usable base,  and give a sign when there is a minimum usable generator. 
 

 

From you i'l take for now 🙂 :

* selecting which API's should be generated or skipped,

* idenifiying identical class.

* ordering classes, automaticly or defined as override

 

Why it takes a little bit time (actual time):
 

* I do this in private (95%)

* Gonna be first "public" repo on github

* Have a ton of things to do at home/garden and garage, 

 

Edited April 19, 2021 by mvanrijnen