Andrea Magni

Hi @Jean Vandromme,

when the client is not a Delphi application it is hard to rely on FireDAC specific features such as delta changes apply mechanism.

I would keep the GET methods as you have written them but I would change the POST methods to something like:

    [POST, Path('/allItems'), Consumes(TMediaType.APPLICATION_JSON)]
    function postAllItems([BodyParam] const AChanges: TJSONObject):TJSONObject;

Then you should go through AChanges JSON object to extract data and apply changes to the DB (using the FD helper object to build some SQL or opening the qItems dataset, making some locate, edit data and post dataset.

This is a very basic approach (not so handy IMHO).

Another, more convenient IMHO, option is to define a record structure mimicking the JSON structure you want to accept as description of data changes and let MARS materialize it from the client's request.

TItem = record
  id: Integer;
  name: string
end;

(...)

    [POST, Path('/allItems'), Consumes(TMediaType.APPLICATION_JSON)]
    function postAllItems([BodyParam] const AChanges: TArray<TItem>): TJSONObject;

Once you have AChanges materialized automatically by MARS, you can take advantage of the TRecord<TItem>.ToDataSet implementation to apply changes to the dataset (beware of locating the right record in advance or query the record using the id for best performance).

Let me know if it is clear enough... If not I can provide some demo to show you what I am suggesting to do.

Sincerely,

Andrea

Hi @Stuart Clennett everything seems fine in your screenshot (as long as you actually loaded a file as param's value but I take this for granted).

It turned out to be a bug in MARS I introduced with commit https://github.com/andrea-magni/MARS/commit/a0bcbce5c43dbb45648d89e6bc3a44b74e7996bd 

Introducing the RequiredAttribute to enforce FormParam validation I broke the reading mechanism of data from multipart/form-data files.

I fixed this in the develop branch with commit https://github.com/andrea-magni/MARS/commit/bf84a00b0f5e45d8760ab9b0a24d7c6ee018aca4

I also backported a similar fix to master branch with commit https://github.com/andrea-magni/MARS/commit/63a4bbb36d4f6acb18106086f889b76ac69f2882

Please update your working copy (either develop or master) and confirm this solves the problem.

Thanks a lot for pointing this out and sorry for the inconvenience.

Sincerely,

Andrea

Hi,

yes, there's a bug in the way MARS defaults the filename for the ini file when deployed as Apache module.

As a workaround you can change this line (in Server.Ignition.pas file):

    FEngine.Parameters.LoadFromIniFile;

to

    FEngine.Parameters.LoadFromIniFile(ChangeFileExt(GetModuleName(HInstance), '.ini'));

If your module is located in /etc/apache2/modules/libMyModerverApacheModule.so, the configuration file would be located in the same folder (/etc/apache2/modules/libMyModerverApacheModule.ini).

Here's an easy way to check if your config file gets actually loaded:

1) add a Test entry in your config file

[DefaultEngine]

DefaultApp.Test=Hello world!

2) add a config function to one of your resources    

[ApplicationParamFunc] MyConfig: TConfigParamFunc;

3) return the value from one of your REST methods:

MyConfig('Test').AsString

I've fixed the issue in develop branch and backported to master as well ( https://github.com/andrea-magni/MARS/commit/576ecbc5b0798fbec430675be1bcff8fc0ef2600 )

Let me know if this solves your problem.

Thanks

Ciao

You are probably missing to actually reply to preflight requests (request made with OPTIONS verb).

Look here: https://github.com/andrea-magni/MARS/blob/master/Demos/MARSTemplate/Server.Ignition.pas

A commented example for FEngine.BeforeHandleRequest is provided and there is a (furtherly) commented section to address preflight requests:

        // Handle CORS and PreFlight
        if SameText(ARequest.Method, 'OPTIONS') then
        begin
          Handled := True;
          Result := False;
        end;

This is a very minimalistic implementation (basically responding 200 OK to every preflight request) but it's needed for CORS to properly work.

HTH,

Andrea

Hi, @Rafael Mattos

The first version is the correct one (it's a Boolean value), if you want to set that parameter via code.

Consider you can easily set CORS options in the default ini file associated with your server application.

Simply add a line under the corresponding engine's section:

[DefaultEngine]

CORS.Enabled=true

You can set these additional options (here listed with their default values), if you need it:

CORS.Origin=*
CORS.Methods=HEAD,GET,PUT,POST,DELETE,OPTIONS
CORS.Headers=X-Requested-With,Content-Type,Authorization

Sincerely,

Andrea

6 hours ago, Stuart Clennett said:

HI Andrea,

Yes, that was it, I had AuthEndorsement set to cookie.

So sorry, I thought I had checked that but clearly not.

No problem at all!

I am investigating if it would be possible to force cookies of the client when loading the token from file/stream. IIRC it was not easy to do with one of the two http client libraries currently available (Indy and NetHttpClient).

Sincerely,

Andrea

Hi, hard to say without looking at the code but I guess it is because of the TMARSClient.AuthEndorsement value.

By default MARS uses cookies to endorse the token in the requests. But if you are loading the token from file, it is quite probable the client does not have the cookie set.

MARS actually supports two ways to endorse the token in the request: cookie and the Authorization header. If you set the client AuthEndorsement property to AuthorizationBearer, does it works?

I've built a simple example (server and client) as an example:

https://www.dropbox.com/s/zu0ovdhj1g88md9/MyConsole.zip?dl=0

Just run the MyConsoleServerApplication and then the MyConsole.

Let me know if everything is fine.

Sincerely,

Andrea

Hi all,

@Rafael Mattos, I am fine thanks! I am relatively silent these weeks as I am a bit busier than usual and time is always lacking. Sorry about that.

MARS collects result values of your methods and use them to properly serialize them and send them to the client.

The result value will be freed by MARS at the end of the "Activation" (the handling of an incoming request down to the sending of the serialized result to the client).

If you want to prevent MARS from freeing the instance you are returning, you can decorate your method with the IsReference attribute ( defined in https://github.com/andrea-magni/MARS/blob/master/Source/MARS.Core.Attributes.pas ).

[GET, PermitAll, IsReference]
    function GetProduct([FormParam] ClientId: Integer):TJSONArray;

Hint: there was a typo in your PermiAll attribute (PermiteAll) and you can omit the PermitAll attribute as it is the default behavior.

Let me know if this solves your problem.

Sincerely,

Hi @kitesoft2015@gmail.com,

you can have a method, in your resource, marked with AfterInvoke attribute.

This method will be called after the actual response has been sent to the client so I think it would be a good moment to delete the file.

You may store the temp file name in a field of your resource.

In theory for temporary files you may ask the operating system to provide a proper file name (and path) so that the OS knows it can clean those files when needed. I guess your question comes because you actually don't want that file to stay on the file system longer than needed.

Also consider you may serve a stream (without passing from the disk file), if you can.

Sincerely,

Andrea

I am not sure to be there, sorry. However if I'd make it, I may consider buying some copies from you to make some gifts for my best customers 😉

Congrats! I just ordered my paperback copy (you'll sign it one day 😉 )

SIncerely

Hi @Ruslan, if you are freeing (closing) the frame (frameinfo) within an event handler of an UI element you may need to wait some amount of time in order to let the UI finish its activities related to the event itself (i.e. if it's a button, there might be some animations going on and/or similar things).

Sincerely,

Andrea

You are welcome. We may consider adding some kind of (light) validation to the serialization/deserialization mechanisms.

I like to keep MARS lightweight but something like "this field is required" could be safe to implement.

Hi Marco, thanks for the kind words about MARS, happy to see you using it.

There are a number of topics involved here as you surely know. Data validation and how to send results to the client have a lot of implications (involving your API design).I will try to expose some capabilities MARS has that may be useful when dealing with this topics but then it's up to you to choose an approach suitable for you on the server side to implement and for the client side to properly (conveniently) handle.

Scenario:

We defined a TMyRecord type as follow:

  TMyRecord = record
    Name: string;
    Surname: string;
    Age: Integer;
    DateOfBirth: TDate;
    function ToString: string;
  end;

function TMyRecord.ToString: string;
begin
  Result := string.join(' ', [Name, SurName, DateToStr(DateOfBirth), Age.ToString]);
end;

and we also have defined a REST method, expecting the client to send a JSON object representing our record:

    [POST, Consumes(TMediaType.APPLICATION_JSON), Produces(TMediaType.TEXT_PLAIN)]
    function SayHelloWorld([BodyParam] ARecord: TMyRecord): string;

*** First approach: hooking into de-serialization mechanism ***

MARS JSON-to-record built in support has a couple of goodies like:

  - you can add an additional "_AssignedValues: TArray<string>" field to your record and MARS will fill that array with the names of the fields of the record actually filled from the provided JSON object (so you can determine if the Age field is zero because it was not provided by the client or has been provided with the zero value)

  - you can add your record a function with the following prototype 

function ToRecordFilter(const AField: TRttiField; const AObj: TJSONObject): Boolean;

MARS JSON-to-record mapping mechanism will call this function for each field of the record so you have a chance to transform or validate input data there.

Example:

    [JSONName('')] // no JSON serialization/deserialization
    ValidationErrors: TArray<string>;


function TMyRecord.ToRecordFilter(const AField: TRttiField;
  const AObj: TJSONObject): Boolean;
begin
  Result := True; // no filtering

  if SameText(AField.Name, 'Age') then
  begin
    if AObj.ReadIntegerValue('Age') < 18 then
      ValidationErrors := ValidationErrors + ['Age: You must be 18+ to enter'];
  end
  else if SameText(AField.Name, 'DateOfBirth') then
  begin
    var LDateOfBirth := AObj.ReadDateTimeValue('DateOfBirth', -1);
    if LDateOfBirth > Now then
      ValidationErrors := ValidationErrors + ['DateOfBirth: Value cannot be in the future']
    else if LDateOfBirth = -1 then
      ValidationErrors := ValidationErrors + ['DateOfBirth: Value is mandatory'];
  end;
end;

You can also return False and manually assign the record field value (using some defaults or transforming data from the AObj JSON object).

So you'll find your ValidationErrors field filled with data you can check whenever you need to know if the record is valid or not. I would add two methods to the record: 

function TMyRecord.IsValid: Boolean;
begin
  Result := Length(ValidationErrors) = 0;
end;

procedure TMyRecord.Validate;
begin
  if not IsValid then
    raise Exception.Create(string.join(sLineBreak, ValidationErrors));
end;

You REST method would be implemented this way:

function THelloWorldResource.SayHelloWorld([BodyParam] ARecord: TMyRecord): string;
begin
  ARecord.Validate; // exception if fails --> 500 Internal server error
  Result := ARecord.ToString;
end;

Pros:

  - you have a single point of validation, automatically called during de-serialization

  - you may eventually follow a symmetrical approach for serialization, handling TMyRecord.ToJSONFilter(const AField: TRttiField; const AObj: TJSONObject): Boolean;

  - you can fine tune what to send to the client if validation fails (let the exception flow to MARS and have a 500 Internal server error or handle it and provide some different message / result / default)

Cons:

  - validation is not compile checked (using field names as strings) and this may lead to maintainability issues on the long term;

  - even though you can take advantage of some AOP technique (i.e attributes on record fields) to generalize some validation rules, you'll need to replicate implementation of ToRecordFilter (and ValidationErrors-like fields) across all records of your data model (50 record types, 50 implementations, probably with some copy-and-paste trouble);

*** Second approach (a variation of the first one) ***

We can try to solve/mitigate the Cons of the previous approach by taking advantage of _AssignedValues field (automatically managed by the de-serialization mechanism) and adding a method collecting all validation rules:

function Validate(out AMessages: TArray<string>): Boolean; overload;
  
function TMyRecord.Validate(out AMessages: TArray<string>): Boolean;
begin
  Result := True;
  AMessages := [];

  if Age < 18 then
  begin
    Result := False;
    AMessages := AMessages + ['Age: You must be 18+ to enter'];
  end;

  if IndexStr('DateOfBirth', _AssignedValues) = -1 then
  begin
    Result := False;
    AMessages := AMessages + ['DateOfBirth: Field is required'];
  end;

  if Result and (YearsBetween(DateOfBirth, Now) <> Age) then
  begin
    Result := False;
    AMessages := AMessages + ['DateOfBirth and Age mismatch'];
  end;
end;

procedure TMyRecord.Validate;
var
  LMessages: TArray<string>;
begin
  LMessages := [];
  if not Validate(LMessages) then
    raise Exception.Create(string.join(sLineBreak, LMessages));
end;

So now, when our REST method calls Validate, validation actually occurs (so it's a bit deferred with respect to the previous approach) and validation is written without using strings to refer to fields (better compile time checking and less maintainability issues).

Cons:

  - JSON raw data is hard to access at validation point. We may try to mix the two approaches or cache the raw JSON string in the record itself (if there are no memory usage concerns).

*** Third approach: validation through readers ***

A completely different approach would be to define a custom reader (inheriting from TRecordReader, MARS.Core.MessageBodyReaders.pas and overriding the ReadFrom method). 

This way you have a single point where you can access:

  - raw JSON data

  - de-serialized record (every kind of record you register the reader for)

  - IMARSActivation instance (basically everything about that invocation, including references to the current resource, method and so on)

If you have several record type to deal with, I would probably go for this approach as you may then define your own custom attributes to define validation rules on your record definitions and enforce them in the ReadFrom method implementation (you have a TValue containing the record, you can inspect that record through Rtti to enumerate fields and their attributes to perform validation).

You can then determine what to do with respect to the client through a corresponding MessageBodyWriter or through one of the error-handling capabilities MARS has (TMARSActivation.RegisterInvokeError or adding a method to your resources with the InvokeError attribute).

The topic is a bit wide to cover every possible scenario. If you can, please specify a bit what you would like to implement / how many data structures you'll need to handle / what kind of approach with respect to the client you'd like to implement (sometime I've seen people willing to return a opaque 500 Internal server error, sometimes willing to return some details about what caused the error, some other wants full control over the response to provide a detailed error message, eventually in a different content type of the original one and so on...).

I will try to help further (sorry for the late I am a bit overwhelmed this weeks).

Sincerely,

Another (small) addition (on the same path):

1) change SanitizeStr to this:

function TForm1.SanitizeStr(const AString: string; const ADelimiter: string): string;
begin
  Result := AString
    .Replace(' ' + ADelimiter, ADelimiter, [rfReplaceAll])
    .Replace(ADelimiter + ' ', ADelimiter, [rfReplaceAll]);
end;

2) change the SourceExpression to 'Owner.SanitizeStr(Text, Owner.ListBox1.Items.Delimiter)'

And you can avoid setting the Items.Delimiter for your listbox.

"It's a small step for man but..." :-)

On 1/12/2019 at 12:05 PM, Primož Gabrijelčič said:

Does anyone see a better solution for that? (Better = requires writing less code.)

Not that I think that this would be a significant improvement with respect to your approach, but you can avoid the OnAssigningValue event-handler by introducing this public function to your form:

function TForm1.SanitizeStr(const AString: string): string;
begin
  Result := AString.Replace(' ,', ',', [rfReplaceAll]).Replace(', ', ',', [rfReplaceAll]);
end;

Then use as SourceExpression of your TBindExpression this value: 'Owner.SanitizeStr(Text)'.

AFAIK "Owner" here stands for the owner of the BindingList and the LB expression engine is capable of calling a method on the underlying object (your form instance).

Same lines of pascal code but one line left out from the XFM file (the one defining the event handler for OnAssigningValue event) and you can declare your function at your will (no prototype to match for the event handler).

Does this match your definition of Better = writing less code? :-)

Sincerely

Basic tutorial on how to enable app shortcuts for your FMX apps.

Delphi 10.3 Rio required (as the app shortcuts are available on Android since version 7.1, API Level 25).

https://blog.andreamagni.eu/2019/02/how-to-add-android-app-shortcuts-to-a-fmx-application/

Sincerely,

Andrea

14 hours ago, hsvandrew said:

Apologies @Andrea Magni I wasn't able to edit the survey once it collected results and I wasn't aware of the MARS REST Library

No worries 🙂

If you need some information about the library, feel free to ask me (there is a dedicated subforum in the third party group).

Sincerely,

Andrea

23 hours ago, hsvandrew said:

This note is for anyone going about creating an ISAPI in modern versions of Delphi. Request.TranslateURI has been broken (and still remains unfixed in 10.3)for a number of releases.

TranslateURI is useful if you run multiple sites on the same IIS server and want to work out the base folder of the site i.e. request.TranslateURI('/')  to work out which customer experience/domain etc you are currently serving.

There's no entry for my MARS REST library (https://github.com/andrea-magni/MARS) in your poll...

I am happily doing web backend development since years, using DataSnap at first. Using MARS since it exists 😉

Hi, everything seems fine, just be sure your call to TMARSResourceRegistry.Instantce.RegisterResource<THealthCheck> takes place *before* the second call to AddApplication (in TServerEngine.CreateEngine method).

I usually make this happen adding the name of the unit where THealthCheck to the uses clause list of the Server.Ignition.pas file (in the MARSTemplate demo, you can spot the Server.Resources unit listed there for the same reason).

I am considering, in a future version of MARS, to switch from this 'initialization based' way of registering resources to a more 'configuration based' one. Still have to think about it and choose a simple approach to this problem.

Sincerely,

Andrea

In most of the cases, it is just a matter of replacing the TMARSClient with TMARSNetClient. Beware of references to the original component (i.e. TMARSApplication or TMARSClientToken). 

Also beware the master branch has a defect for 10.3 Rio packages (fixed in the develop branch). 

Sincerely 

3 hours ago, MrMe said:

Hi Andrea!

I'm glad to hear that you support Rio. Is there any plan to replace Indy on the client side?

 

Apple doesn't support OpenSSL for iOS anymore.

Hi, there are two implementation available TMARSIndyClient and TMARSNetClient. 

TMARSClient is actually an alias for TMARSIndyClient, to preserve backward compatibility but you can freely decide to use TMARSNetClient (as I do most of the times). 

3 hours ago, MrMe said:

And thank you for this very great library

Thanks for the kind words! 

Sincerely, Andrea

Hi @Stuart Clennett,

you can easily provide values for JWT configuration parameters through the server application configuration.

MARSTemplate based applications looks for an ini file (same name of the server application) structured this way:

[YourEngineName]
YourAppName.YourParam=YourValue

So, if you just cloned MARSTemplate with MARSCmd utility, you can change JWT secret and issuer this way:

[DefaultEngine]
DefaultApp.JWT.Secret=Andrea123
DefaultApp.JWT.Issuer=Andrea

Beware if you change the engine name (form DefaultEngine) or the application name (from DefaultApp) you'll need to correct your ini file.

Check you Server.Ignition.pas file and locate this line:     FEngine.Parameters.LoadFromIniFile;

This is where MARS tries to load configuration from the INI file.

If you want to hard-code (or read from different sources) parameters values you can either add this line, keeping in mind you are actually defining a value for a parameter named 'JWT.Issuer' of an application named 'DefaultApp' (and the application name has to match):

    FEngine.Parameters.LoadFromIniFile;
    FEngine.Parameters.Values['DefaultApp.JWT.Issuer'] := 'My Issuer';

Or directly define the 'JWT.Issuer' (or any other parameter) to the configuration of the specific application:

    // Application configuration
    LApp := FEngine.AddApplication('DefaultApp', '/default', [ 'Server.Resources.*']);
    LApp.Parameters.Values['JWT.Issuer'] := 'My Issuer';

There is no much difference between these two options, but the second one may be handy if the name of the application is not a constant.

A list of available JWT-related parameters (and their default values) is available here: https://github.com/andrea-magni/MARS/blob/master/Source/MARS.Utils.JWT.pas

Sorry for the late (holiday time plus I've been sick too). 

Sincerely,

Andrea

1 hour ago, Stuart Clennett said:

Thanks for sorting this.  I do have a github account (stuartclennett), but I'm not very familiar with git (aside from commit - push - checkout), so I wouldn't be very confident in dealing with pull-requests as yet.  Thanks for mentioning me though.

It's easy, basically:

1) fork your copy of MARS repository (stuartclennet/MARS)

2) clone that instead of mine (andrea-magni/MARS)

3) commit your changes there and push to your GitHub repository

4) create a Pull Request (from the GitHub web interface or using other tools, I tend to use the web interface or the TortoiseGit functionalities) and I will have a chance to review your PR and merge it to the official MARS repository

This way your contribution will show up both in your account and in the contributors list of MARS official repository.

1 hour ago, Stuart Clennett said:

Should I add any bugs to the github issues directly rather than here in future?

As you like, I get notified by email when someone post here as well when someone opens a new issue on github.

If it is clearly a bug, then I woul open a github issue, if there's room for discussion maybe I would post here.

Thanks.

Sincerely,

Andrea

On 12/16/2018 at 1:29 AM, Alberto Miola said:

Hiandrea,

 

congratulations it's a very nice project! I didn't know about the existance of MARS, I am sure that I'm going to use it next year for a REST project that I have to do at work. I'd like to ask you: is there a component for this library? Or I simply have to import the .pas files?

Ciao Alberto,

thanks for the kind words and I hope you'll enjoy MARS. Feel free to ask questions here or open issues on GitHub if you run into troubles.

MARS is a library suitable for creating REST Server projects and also has a client library to build REST Client projects. You can use them independently one from the other (this means MARS REST servers can be 100% Delphi agnostics and this also stands for the Client library that can consume any REST service out there) or use them together and get some Delphi-to-Delphi specific advantages (like some FireDAC integrations).

Installation instructions are available here: https://github.com/andrea-magni/MARS/blob/master/docs/Installation.md

Let me know if you need help or if you have questions.

Sincerely,