Jump to content

Andrea Magni

Members
  • Content Count

    59
  • Joined

  • Last visited

Everything posted by Andrea Magni

  1. You are welcome. We may consider adding some kind of (light) validation to the serialization/deserialization mechanisms. I like to keep MARS lightweight but something like "this field is required" could be safe to implement.
  2. Hi Marco, thanks for the kind words about MARS, happy to see you using it. There are a number of topics involved here as you surely know. Data validation and how to send results to the client have a lot of implications (involving your API design).I will try to expose some capabilities MARS has that may be useful when dealing with this topics but then it's up to you to choose an approach suitable for you on the server side to implement and for the client side to properly (conveniently) handle. Scenario: We defined a TMyRecord type as follow: TMyRecord = record Name: string; Surname: string; Age: Integer; DateOfBirth: TDate; function ToString: string; end; function TMyRecord.ToString: string; begin Result := string.join(' ', [Name, SurName, DateToStr(DateOfBirth), Age.ToString]); end; and we also have defined a REST method, expecting the client to send a JSON object representing our record: [POST, Consumes(TMediaType.APPLICATION_JSON), Produces(TMediaType.TEXT_PLAIN)] function SayHelloWorld([BodyParam] ARecord: TMyRecord): string; *** First approach: hooking into de-serialization mechanism *** MARS JSON-to-record built in support has a couple of goodies like: - you can add an additional "_AssignedValues: TArray<string>" field to your record and MARS will fill that array with the names of the fields of the record actually filled from the provided JSON object (so you can determine if the Age field is zero because it was not provided by the client or has been provided with the zero value) - you can add your record a function with the following prototype function ToRecordFilter(const AField: TRttiField; const AObj: TJSONObject): Boolean; MARS JSON-to-record mapping mechanism will call this function for each field of the record so you have a chance to transform or validate input data there. Example: [JSONName('')] // no JSON serialization/deserialization ValidationErrors: TArray<string>; function TMyRecord.ToRecordFilter(const AField: TRttiField; const AObj: TJSONObject): Boolean; begin Result := True; // no filtering if SameText(AField.Name, 'Age') then begin if AObj.ReadIntegerValue('Age') < 18 then ValidationErrors := ValidationErrors + ['Age: You must be 18+ to enter']; end else if SameText(AField.Name, 'DateOfBirth') then begin var LDateOfBirth := AObj.ReadDateTimeValue('DateOfBirth', -1); if LDateOfBirth > Now then ValidationErrors := ValidationErrors + ['DateOfBirth: Value cannot be in the future'] else if LDateOfBirth = -1 then ValidationErrors := ValidationErrors + ['DateOfBirth: Value is mandatory']; end; end; You can also return False and manually assign the record field value (using some defaults or transforming data from the AObj JSON object). So you'll find your ValidationErrors field filled with data you can check whenever you need to know if the record is valid or not. I would add two methods to the record: function TMyRecord.IsValid: Boolean; begin Result := Length(ValidationErrors) = 0; end; procedure TMyRecord.Validate; begin if not IsValid then raise Exception.Create(string.join(sLineBreak, ValidationErrors)); end; You REST method would be implemented this way: function THelloWorldResource.SayHelloWorld([BodyParam] ARecord: TMyRecord): string; begin ARecord.Validate; // exception if fails --> 500 Internal server error Result := ARecord.ToString; end; Pros: - you have a single point of validation, automatically called during de-serialization - you may eventually follow a symmetrical approach for serialization, handling TMyRecord.ToJSONFilter(const AField: TRttiField; const AObj: TJSONObject): Boolean; - you can fine tune what to send to the client if validation fails (let the exception flow to MARS and have a 500 Internal server error or handle it and provide some different message / result / default) Cons: - validation is not compile checked (using field names as strings) and this may lead to maintainability issues on the long term; - even though you can take advantage of some AOP technique (i.e attributes on record fields) to generalize some validation rules, you'll need to replicate implementation of ToRecordFilter (and ValidationErrors-like fields) across all records of your data model (50 record types, 50 implementations, probably with some copy-and-paste trouble); *** Second approach (a variation of the first one) *** We can try to solve/mitigate the Cons of the previous approach by taking advantage of _AssignedValues field (automatically managed by the de-serialization mechanism) and adding a method collecting all validation rules: function Validate(out AMessages: TArray<string>): Boolean; overload; function TMyRecord.Validate(out AMessages: TArray<string>): Boolean; begin Result := True; AMessages := []; if Age < 18 then begin Result := False; AMessages := AMessages + ['Age: You must be 18+ to enter']; end; if IndexStr('DateOfBirth', _AssignedValues) = -1 then begin Result := False; AMessages := AMessages + ['DateOfBirth: Field is required']; end; if Result and (YearsBetween(DateOfBirth, Now) <> Age) then begin Result := False; AMessages := AMessages + ['DateOfBirth and Age mismatch']; end; end; procedure TMyRecord.Validate; var LMessages: TArray<string>; begin LMessages := []; if not Validate(LMessages) then raise Exception.Create(string.join(sLineBreak, LMessages)); end; So now, when our REST method calls Validate, validation actually occurs (so it's a bit deferred with respect to the previous approach) and validation is written without using strings to refer to fields (better compile time checking and less maintainability issues). Cons: - JSON raw data is hard to access at validation point. We may try to mix the two approaches or cache the raw JSON string in the record itself (if there are no memory usage concerns). *** Third approach: validation through readers *** A completely different approach would be to define a custom reader (inheriting from TRecordReader, MARS.Core.MessageBodyReaders.pas and overriding the ReadFrom method). This way you have a single point where you can access: - raw JSON data - de-serialized record (every kind of record you register the reader for) - IMARSActivation instance (basically everything about that invocation, including references to the current resource, method and so on) If you have several record type to deal with, I would probably go for this approach as you may then define your own custom attributes to define validation rules on your record definitions and enforce them in the ReadFrom method implementation (you have a TValue containing the record, you can inspect that record through Rtti to enumerate fields and their attributes to perform validation). You can then determine what to do with respect to the client through a corresponding MessageBodyWriter or through one of the error-handling capabilities MARS has (TMARSActivation.RegisterInvokeError or adding a method to your resources with the InvokeError attribute). The topic is a bit wide to cover every possible scenario. If you can, please specify a bit what you would like to implement / how many data structures you'll need to handle / what kind of approach with respect to the client you'd like to implement (sometime I've seen people willing to return a opaque 500 Internal server error, sometimes willing to return some details about what caused the error, some other wants full control over the response to provide a detailed error message, eventually in a different content type of the original one and so on...). I will try to help further (sorry for the late I am a bit overwhelmed this weeks). Sincerely,
  3. Impressive @RDPasqua! I can see a 8x speed up running a MARS "Hello World" project with 10K requests and a concurrency level of 100 (through Apache benchmark on localhost). It passes from 59.842 seconds to 7.617 (shrinking down total execution time to 12,7%). Of course my benchmark scenario can be improved (a Win10 VMWare Fusion machine, dual core over my MacBookPro late 2017, i7@3.5GHz). I made another test, querying a local Firebird (32bit) instance and performing a 'select * from EMPLOYEE' query with FireDAC to a (non-FireDAC) JSON serialization (about 10K bytes) and it passes from 33.640 seconds to 29.951 (shrinking down total execution time to 89%). So this time the gain is limited but still a good 10%. I am actually a bit puzzled how it's possible the first benchmark (textual 'Hello, World!') to be 200% slower than the second (returning a 42 records dataset JSON representation) but that has nothing to do with your memory manager replacement (although using RDPMM64 this does not happen anymore). I am attaching ab results for both scenarios. And on my MARS's to-do list now there's an entry to integrate Delphi-Cross-Socket as http server. 🙂 Congrats and keep up the good work! Andrea Benchmark_HelloWorld_DB_JSON_Response_Win64.txt Benchmark_HelloWorld_Text_Response_Win64.txt
  4. Andrea Magni

    Generating one-to-many on the fly with Live Bindings

    Another (small) addition (on the same path): 1) change SanitizeStr to this: function TForm1.SanitizeStr(const AString: string; const ADelimiter: string): string; begin Result := AString .Replace(' ' + ADelimiter, ADelimiter, [rfReplaceAll]) .Replace(ADelimiter + ' ', ADelimiter, [rfReplaceAll]); end; 2) change the SourceExpression to 'Owner.SanitizeStr(Text, Owner.ListBox1.Items.Delimiter)' And you can avoid setting the Items.Delimiter for your listbox. "It's a small step for man but..." :-)
  5. Andrea Magni

    Generating one-to-many on the fly with Live Bindings

    Not that I think that this would be a significant improvement with respect to your approach, but you can avoid the OnAssigningValue event-handler by introducing this public function to your form: function TForm1.SanitizeStr(const AString: string): string; begin Result := AString.Replace(' ,', ',', [rfReplaceAll]).Replace(', ', ',', [rfReplaceAll]); end; Then use as SourceExpression of your TBindExpression this value: 'Owner.SanitizeStr(Text)'. AFAIK "Owner" here stands for the owner of the BindingList and the LB expression engine is capable of calling a method on the underlying object (your form instance). Same lines of pascal code but one line left out from the XFM file (the one defining the event handler for OnAssigningValue event) and you can declare your function at your will (no prototype to match for the event handler). Does this match your definition of Better = writing less code? :-) Sincerely
  6. Basic tutorial on how to enable app shortcuts for your FMX apps. Delphi 10.3 Rio required (as the app shortcuts are available on Android since version 7.1, API Level 25). https://blog.andreamagni.eu/2019/02/how-to-add-android-app-shortcuts-to-a-fmx-application/ Sincerely, Andrea
  7. I just pushed some changes to the TFrameStand GitHub.com repository (https://github.com/andrea-magni/TFrameStand) in order to support 10.3 Rio version. This release is TFrameStand 1.4 ( https://github.com/andrea-magni/TFrameStand/releases/tag/v.1.4 ) and will be available through GetIt ASAP. Sincerely
  8. Andrea Magni

    ISAPI in Delphi 10.2,10.3 TranslateURI

    No worries 🙂 If you need some information about the library, feel free to ask me (there is a dedicated subforum in the third party group). Sincerely, Andrea
  9. Andrea Magni

    ISAPI in Delphi 10.2,10.3 TranslateURI

    There's no entry for my MARS REST library (https://github.com/andrea-magni/MARS) in your poll... I am happily doing web backend development since years, using DataSnap at first. Using MARS since it exists 😉
  10. Andrea Magni

    Adding a second app to a MARS Engine

    Hi, everything seems fine, just be sure your call to TMARSResourceRegistry.Instantce.RegisterResource<THealthCheck> takes place *before* the second call to AddApplication (in TServerEngine.CreateEngine method). I usually make this happen adding the name of the unit where THealthCheck to the uses clause list of the Server.Ignition.pas file (in the MARSTemplate demo, you can spot the Server.Resources unit listed there for the same reason). I am considering, in a future version of MARS, to switch from this 'initialization based' way of registering resources to a more 'configuration based' one. Still have to think about it and choose a simple approach to this problem. Sincerely, Andrea
  11. Changes to support Delphi 10.3 Rio are now in the official repository ( https://github.com/andrea-magni/MARS ). More changes will follow in the near future 🙂 Enjoy, Andrea
  12. Andrea Magni

    MARS now supports Delphi 10.3 Rio :-)

    In most of the cases, it is just a matter of replacing the TMARSClient with TMARSNetClient. Beware of references to the original component (i.e. TMARSApplication or TMARSClientToken). Also beware the master branch has a defect for 10.3 Rio packages (fixed in the develop branch). Sincerely
  13. Andrea Magni

    MARS now supports Delphi 10.3 Rio :-)

    Hi, there are two implementation available TMARSIndyClient and TMARSNetClient. TMARSClient is actually an alias for TMARSIndyClient, to preserve backward compatibility but you can freely decide to use TMARSNetClient (as I do most of the times). Thanks for the kind words! Sincerely, Andrea
  14. Hi @Stuart Clennett, you can easily provide values for JWT configuration parameters through the server application configuration. MARSTemplate based applications looks for an ini file (same name of the server application) structured this way: [YourEngineName] YourAppName.YourParam=YourValue So, if you just cloned MARSTemplate with MARSCmd utility, you can change JWT secret and issuer this way: [DefaultEngine] DefaultApp.JWT.Secret=Andrea123 DefaultApp.JWT.Issuer=Andrea Beware if you change the engine name (form DefaultEngine) or the application name (from DefaultApp) you'll need to correct your ini file. Check you Server.Ignition.pas file and locate this line: FEngine.Parameters.LoadFromIniFile; This is where MARS tries to load configuration from the INI file. If you want to hard-code (or read from different sources) parameters values you can either add this line, keeping in mind you are actually defining a value for a parameter named 'JWT.Issuer' of an application named 'DefaultApp' (and the application name has to match): FEngine.Parameters.LoadFromIniFile; FEngine.Parameters.Values['DefaultApp.JWT.Issuer'] := 'My Issuer'; Or directly define the 'JWT.Issuer' (or any other parameter) to the configuration of the specific application: // Application configuration LApp := FEngine.AddApplication('DefaultApp', '/default', [ 'Server.Resources.*']); LApp.Parameters.Values['JWT.Issuer'] := 'My Issuer'; There is no much difference between these two options, but the second one may be handy if the name of the application is not a constant. A list of available JWT-related parameters (and their default values) is available here: https://github.com/andrea-magni/MARS/blob/master/Source/MARS.Utils.JWT.pas Sorry for the late (holiday time plus I've been sick too). Sincerely, Andrea
  15. It's easy, basically: 1) fork your copy of MARS repository (stuartclennet/MARS) 2) clone that instead of mine (andrea-magni/MARS) 3) commit your changes there and push to your GitHub repository 4) create a Pull Request (from the GitHub web interface or using other tools, I tend to use the web interface or the TortoiseGit functionalities) and I will have a chance to review your PR and merge it to the official MARS repository This way your contribution will show up both in your account and in the contributors list of MARS official repository. As you like, I get notified by email when someone post here as well when someone opens a new issue on github. If it is clearly a bug, then I woul open a github issue, if there's room for discussion maybe I would post here. Thanks. Sincerely, Andrea
  16. Andrea Magni

    MARS now supports Delphi 10.3 Rio :-)

    Ciao Alberto, thanks for the kind words and I hope you'll enjoy MARS. Feel free to ask questions here or open issues on GitHub if you run into troubles. MARS is a library suitable for creating REST Server projects and also has a client library to build REST Client projects. You can use them independently one from the other (this means MARS REST servers can be 100% Delphi agnostics and this also stands for the Client library that can consume any REST service out there) or use them together and get some Delphi-to-Delphi specific advantages (like some FireDAC integrations). Installation instructions are available here: https://github.com/andrea-magni/MARS/blob/master/docs/Installation.md Let me know if you need help or if you have questions. Sincerely,
  17. Thanks again for spotting this bug. If you like your contribution to this project to be more visible, please set up a GitHub account (if you haven't yet) and add a pull request for this (or future) corrections. I can obviously apply your changes to MARS but your contribution will not be tracked by GitHub (even if I always try to mention the author in the commit message in this situations). I have merged your changes with this commit ( https://github.com/andrea-magni/MARS/commit/9f45d60e7adb65a36432840baf2cf3e526a8f518 ) and also fixed LastCmdSuccess method. I went for returning -1 in ResponseStatusCode when FLastResponse is not available (rather than zero). BTW I didn't know about Charles proxy ( https://www.charlesproxy.com/ ) I had some troubles configuring WireShark and Fiddler inside a VM but I can see Charles runs perfectly with no clue. Seems very nice so far, thanks!
  18. Andrea Magni

    Custom media type

    Apart from the registration order, you can define a higher affinity of your writer in order to supersede the standard ones. Just switch the value returned by the affinity function in the RegisterWriter call from TMARSMessageBodyRegistry.AFFINITY_MEDIUM to TMARSMessageBodyRegistry.AFFINITY_HIGH and your writer will have a priority over the standard record writer. I would then write the WriteTo implementation like this: procedure TScimPlusJSONWriter.WriteTo(const AValue: TValue; const AMediaType: TMediaType; AOutputStream: TStream; const AActivation: IMARSActivation); var LType: TRttiType; begin LType := AActivation.MethodReturnType; if not Assigned(LType) then Exit; // procedure? if LType.IsRecord or LType.IsDynamicArrayOfRecord then TMARSMessageBodyWriter.WriteWith<TRecordWriter>(AValue, AMediaType, AOutputStream, AActivation) else if LType.IsObjectOfType<TJSONValue> then TMARSMessageBodyWriter.WriteWith<TJSONValueWriter>(AValue, AMediaType, AOutputStream, AActivation); AActivation.Response.ContentEncoding := 'UTF-8'; end; TMARSMessageBodyWriter.WriteWith<> is a new utility function I just introduced in the "develop" branch (MARS.Core.MessageBodyWriter unit, https://github.com/andrea-magni/MARS/commit/e818d7e261e7ef2b9a1c2c714adae459c5585c56#diff-97e07c61cdc46902bf53d808b8117104R157 ). You can simply instantiate the corresponding MessageBodyWriter class as we did before. Sincerely,
  19. Hi @Stuart Clennett, it's a bug! 🙂 Thanks for spotting it. The TMARSFireDAC.InjectMacroValues and TMARSFireDAC.InjectParamValues implementations assume every TFDAdaptedDataSet actually has a Command (not nil). TFDMemTable do not have Command assigned. Workaround: if you do not need your memory table to be included as the result of Retrieve method, you can decorate it with the RESTExclude attribute: [RESTExclude] FDMemTable1: TFDMemTable; However, I just fixed this in the "develop" branch of MARS repository: https://github.com/andrea-magni/MARS/commit/b6299926671b00e75981c47a74375d9c51c529ca Another workaround would be to change your copy of TMARSFDDataModuleResource.Retrieve and change this line: if LDataSet is TFDAdaptedDataSet then to this: if (LDataSet is TFDAdaptedDataSet) and Assigned(TFDAdaptedDataSet(LDataset).Command) then The MARS.Data.FireDAC.DataModule has nothing special, it is provided as an example of use, you can copy it, change it... it is obviously the counter-part of a TMARSFDResource but there no direct deal with the specific class, just matter of the shape of its methods (types returned, GET/POST selection, ...). Thanks, Andrea
  20. Andrea Magni

    Custom media type

    Hi @Bjørn Larsen, You have a number of possibilities and I would choose one way or another depending on the actual use case. If you need a single (or just a few) methods to produce application/scim+json content type, I would probably follow the way to let MARS produce standard JSON and then fix the ContentType/ContentEncoding. There's a handy way to do this through the Context attribute (MARS will inject a reference to the Response object you will be able to use it across all the methods you need): [Path('helloworld')] THelloWorldResource = class protected [Context] Response: TWebResponse; // add unit Web.HTTPApp to uses clause public [GET, Produces(TMediaType.APPLICATION_JSON)] function User1: TJSONObject; end; (...) function THelloWorldResource.User1: TJSONObject; begin Result := TJSONObject.Create; Result.WriteStringValue('id', TGUID.NewGuid.ToString); Response.ContentType := 'application/scim+json'; Response.ContentEncoding := 'UTF-8'; end; If you have several methods (in a single resource) that need to produce application/scim+json and you want to generalize this "response patching" one for all, you may take advantage of another MARS functionality: AfterInvoke code execution. It comes in a couple of variations, the first one will let you add a method (PatchResponse) and decorate it in order to have MARS execute it after each request to that resource: [Path('helloworld')] THelloWorldResource = class protected [Context] Response: TWebResponse; public [GET, Produces(TMediaType.APPLICATION_JSON)] function User1: TJSONObject; [AfterInvoke] procedure PatchResponse; end; (...) procedure THelloWorldResource.PatchResponse; begin Response.ContentType := 'application/scim+json'; Response.ContentEncoding := 'UTF-8'; end; function THelloWorldResource.User1: TJSONObject; begin Result := TJSONObject.Create; Result.WriteStringValue('id', TGUID.NewGuid.ToString); end; This way, your User1 (and other) method will not be polluted by the "response patching" and at the same time you have a single copy of the "response patching" code (DRY principle). Another AfterInvoke possibility is to define it at a more general level by calling TMARSActivation.RegisterAfterInvoke: TMARSActivation.RegisterAfterInvoke( procedure (const AActivation: IMARSActivation) begin if AActivation.ResourceInstance is THelloWorldResource then // or whatever other strategy begin AActivation.Response.ContentType := 'application/scim+json'; AActivation.Response.ContentEncoding := 'UTF-8'; end; end ); This has the advantage to be very wide (possibly spanning over each MARS engine/application) and does not require the injection of the Response object at resource level. You can setup your own strategy to determine when it's the case to patch the response or not (i.e. use some custom attribute yourself to decorate resources/methods or check the current ContentType or whatever... the AActivation parameter will provide you a very detailed context over the current MARS activation including Request, Response, selected class, selected method, method return value and so on). These are all "patching" options. There is another option available though: define a new media type. From the example you posted, I can't see if you are returning standard or custom types (i.e. TJSONObject instances or TYourClass instances/records). In both cases you can define a new MBW (MessageBodyWriter) for a new media type (application/scim+json) and have full control over Response generation. It's easy, I am attaching here a unit (MBW.ScimPlusJSON.pas) with a new MBW implementation for application/scim+json that will match any request with a Produces('application/scim+json') attribute and will act the same of the standard JSON MBW with the addition of setting the ContentEncoding of the Response. (ContenType will automatically be set by MARS). Just be sure to include the unit in the uses list of your resources unit or in the Server.Ignition unit. You resource then will look like this: [Path('helloworld')] THelloWorldResource = class protected [Context] Response: TWebResponse; public [GET, Produces(MEDIATYPE_APPLICATION_SCIM_PLUS_JSON)] function User1: TJSONObject; end; (...) function THelloWorldResource.User1: TJSONObject; begin Result := TJSONObject.Create; Result.WriteStringValue('id', TGUID.NewGuid.ToString); end; Hope this helps you to solve your problem and please let me know if something is unclear or you need anything else. Sincerely, Andrea MBW.ScimPlusJSON.pas
  21. Andrea Magni

    Error during Execute in MARSCmd

    Thanks for figuring this out! I would say you should not have that file there (as it is not there in the repo https://github.com/andrea-magni/MARS/tree/master/Demos/MARSTemplate/bin ). You probably have it because you actually compiled MARSTemplate demo itself (did you?)... However it is obviously a fault of my strategy to rename files... I will try to improve (probably managing a list of files to rename and the expected final name and then have a much smarter copy strategy). Thanks again for taking the time to spot it and make it evident. I have added an issue on GitHub: https://github.com/andrea-magni/MARS/issues/52 Sincerely, Andrea
  22. Andrea Magni

    Token/Roles not working with TMARSNetClient

    I am starting to think it's something related to Delphi version (Berlin). There are entries in quality.embarcadero.com about TNetHTTPClient and I found this: https://quality.embarcadero.com/browse/RSP-14301 If the client is not sending cookies (and AuthEndorsement is set to Cookie), your situation is explained. The fact it works on my side may depend this may not affect all Berlin versions (I am using Delphi 10.1 Berlin Version 24.0.25048.9432, Berlin update 2 anniversary)... Do you have a chance to check with other Delphi versions? Or maybe someone else with Berlin may try to reproduce this? Thanks
  23. Andrea Magni

    TFrameStand v.1.4: 10.3 Rio support

    BTW, TFrameStand v1.4 is now available through GetIt package manager for 10.3 Rio 🙂
  24. Andrea Magni

    TFrameStand v.1.4: 10.3 Rio support

    Hi Olivier, will have a closer look at your project ASAP. Thanks
  25. I've just released MARS version 1.3 ( https://github.com/andrea-magni/MARS/releases/tag/v1.3 … ). New developments will take place in the new 'develop' branch and there's already a small new feature: basic IXMLDocument support in MBR and MBW. You can consider using the released version in order to avoid unnecessary noise due to new developments, that from now on will take place in a separate 'develop' branch. Sincerely,
×