Best way to validate posted data and return validation errors

[Marco Breveglieri 1]

Hi,

I am (happily) using MARS in a Delphi REST server application.

 

The client has faculty of posting new resource data to the server, and I use a plain record structure to collect it leveraging the [BodyParam] attribute, as seen in the "ToDoList" demo bundled with the product.

 

Now I have to apply some validation and return the expected Bad Request error when some field have been set to an invalid value.

 

Is there some ready-to-use attribute or tool in MARS Curiosity in order to cover this use case?

If not, is there any preferrable way to handle and return one or more validation errors?

 

Thanks in advance for any suggestion, and thanks a lot to Andrea for the really good and useful library!

[Andrea Magni 75]

Hi Marco, thanks for the kind words about MARS, happy to see you using it.

 

There are a number of topics involved here as you surely know. Data validation and how to send results to the client have a lot of implications (involving your API design).I will try to expose some capabilities MARS has that may be useful when dealing with this topics but then it's up to you to choose an approach suitable for you on the server side to implement and for the client side to properly (conveniently) handle.

 

Scenario:

We defined a TMyRecord type as follow:

  TMyRecord = record
    Name: string;
    Surname: string;
    Age: Integer;
    DateOfBirth: TDate;
    function ToString: string;
  end;

function TMyRecord.ToString: string;
begin
  Result := string.join(' ', [Name, SurName, DateToStr(DateOfBirth), Age.ToString]);
end;

and we also have defined a REST method, expecting the client to send a JSON object representing our record:

    [POST, Consumes(TMediaType.APPLICATION_JSON), Produces(TMediaType.TEXT_PLAIN)]
    function SayHelloWorld([BodyParam] ARecord: TMyRecord): string;

 

*** First approach: hooking into de-serialization mechanism ***

 

MARS JSON-to-record built in support has a couple of goodies like:

  - you can add an additional "_AssignedValues: TArray<string>" field to your record and MARS will fill that array with the names of the fields of the record actually filled from the provided JSON object (so you can determine if the Age field is zero because it was not provided by the client or has been provided with the zero value)

  - you can add your record a function with the following prototype 

function ToRecordFilter(const AField: TRttiField; const AObj: TJSONObject): Boolean;

MARS JSON-to-record mapping mechanism will call this function for each field of the record so you have a chance to transform or validate input data there.

Example:

    [JSONName('')] // no JSON serialization/deserialization
    ValidationErrors: TArray<string>;


function TMyRecord.ToRecordFilter(const AField: TRttiField;
  const AObj: TJSONObject): Boolean;
begin
  Result := True; // no filtering

  if SameText(AField.Name, 'Age') then
  begin
    if AObj.ReadIntegerValue('Age') < 18 then
      ValidationErrors := ValidationErrors + ['Age: You must be 18+ to enter'];
  end
  else if SameText(AField.Name, 'DateOfBirth') then
  begin
    var LDateOfBirth := AObj.ReadDateTimeValue('DateOfBirth', -1);
    if LDateOfBirth > Now then
      ValidationErrors := ValidationErrors + ['DateOfBirth: Value cannot be in the future']
    else if LDateOfBirth = -1 then
      ValidationErrors := ValidationErrors + ['DateOfBirth: Value is mandatory'];
  end;
end;

You can also return False and manually assign the record field value (using some defaults or transforming data from the AObj JSON object).

So you'll find your ValidationErrors field filled with data you can check whenever you need to know if the record is valid or not. I would add two methods to the record: 

function TMyRecord.IsValid: Boolean;
begin
  Result := Length(ValidationErrors) = 0;
end;

procedure TMyRecord.Validate;
begin
  if not IsValid then
    raise Exception.Create(string.join(sLineBreak, ValidationErrors));
end;

You REST method would be implemented this way:

function THelloWorldResource.SayHelloWorld([BodyParam] ARecord: TMyRecord): string;
begin
  ARecord.Validate; // exception if fails --> 500 Internal server error
  Result := ARecord.ToString;
end;

 

Pros:

  - you have a single point of validation, automatically called during de-serialization

  - you may eventually follow a symmetrical approach for serialization, handling TMyRecord.ToJSONFilter(const AField: TRttiField; const AObj: TJSONObject): Boolean;

  - you can fine tune what to send to the client if validation fails (let the exception flow to MARS and have a 500 Internal server error or handle it and provide some different message / result / default)

Cons:

  - validation is not compile checked (using field names as strings) and this may lead to maintainability issues on the long term;

  - even though you can take advantage of some AOP technique (i.e attributes on record fields) to generalize some validation rules, you'll need to replicate implementation of ToRecordFilter (and ValidationErrors-like fields) across all records of your data model (50 record types, 50 implementations, probably with some copy-and-paste trouble);

 

*** Second approach (a variation of the first one) ***

We can try to solve/mitigate the Cons of the previous approach by taking advantage of _AssignedValues field (automatically managed by the de-serialization mechanism) and adding a method collecting all validation rules:

function Validate(out AMessages: TArray<string>): Boolean; overload;
  
function TMyRecord.Validate(out AMessages: TArray<string>): Boolean;
begin
  Result := True;
  AMessages := [];

  if Age < 18 then
  begin
    Result := False;
    AMessages := AMessages + ['Age: You must be 18+ to enter'];
  end;

  if IndexStr('DateOfBirth', _AssignedValues) = -1 then
  begin
    Result := False;
    AMessages := AMessages + ['DateOfBirth: Field is required'];
  end;

  if Result and (YearsBetween(DateOfBirth, Now) <> Age) then
  begin
    Result := False;
    AMessages := AMessages + ['DateOfBirth and Age mismatch'];
  end;
end;

procedure TMyRecord.Validate;
var
  LMessages: TArray<string>;
begin
  LMessages := [];
  if not Validate(LMessages) then
    raise Exception.Create(string.join(sLineBreak, LMessages));
end;

 

So now, when our REST method calls Validate, validation actually occurs (so it's a bit deferred with respect to the previous approach) and validation is written without using strings to refer to fields (better compile time checking and less maintainability issues).

Cons:

  - JSON raw data is hard to access at validation point. We may try to mix the two approaches or cache the raw JSON string in the record itself (if there are no memory usage concerns).

 

*** Third approach: validation through readers ***

A completely different approach would be to define a custom reader (inheriting from TRecordReader, MARS.Core.MessageBodyReaders.pas and overriding the ReadFrom method). 

This way you have a single point where you can access:

  - raw JSON data

  - de-serialized record (every kind of record you register the reader for)

  - IMARSActivation instance (basically everything about that invocation, including references to the current resource, method and so on)

If you have several record type to deal with, I would probably go for this approach as you may then define your own custom attributes to define validation rules on your record definitions and enforce them in the ReadFrom method implementation (you have a TValue containing the record, you can inspect that record through Rtti to enumerate fields and their attributes to perform validation).

You can then determine what to do with respect to the client through a corresponding MessageBodyWriter or through one of the error-handling capabilities MARS has (TMARSActivation.RegisterInvokeError or adding a method to your resources with the InvokeError attribute).

 

The topic is a bit wide to cover every possible scenario. If you can, please specify a bit what you would like to implement / how many data structures you'll need to handle / what kind of approach with respect to the client you'd like to implement (sometime I've seen people willing to return a opaque 500 Internal server error, sometimes willing to return some details about what caused the error, some other wants full control over the response to provide a detailed error message, eventually in a different content type of the original one and so on...).

I will try to help further (sorry for the late I am a bit overwhelmed this weeks).

 

Sincerely,

[Marco Breveglieri 1]

Hi Andrea,

thank you very much for all the information provided, really far more than I hoped for.

 

I only need to verify the presence of some mandatory fields and the range of some values, nothing complicated.

 

My question originated from the fact that I am used to validation mechanisms embedded in other frameworks, such as ASP.NET MVC, and so I wanted to check if there was something ready to use inside MARS that would provide the same benefits, e.g. field attributes or similar.

 

I think the second implementation is a good balance between clarity and effectiveness so I will implement my validation code following that sample.

 

Thanks again for your great support!

  Marco.

[Andrea Magni 75]

You are welcome. We may consider adding some kind of (light) validation to the serialization/deserialization mechanisms.

I like to keep MARS lightweight but something like "this field is required" could be safe to implement.