MacOS AVPlayer and DRM

[CHackbart 13]

Hi,

I'm not sure if I did something wrong, but it is nearly impossible to debug - since the application is crashing without any further error message after providing the necessary information to the AVContentKeySession. Apples documentation is also quite rudimentary when it comes to proper handling of DRM content. In theory you create a content key session:

FContentKeySession := TAVContentKeySession.Wrap(TAVContentKeySession.OCClass.contentKeySessionWithKeySystem (AVContentKeySystemFairPlayStreaming)); // FairPlayStreaming

Next you implement your own implementation of AVContentKeySessionDelegate which handles receiving the certificate and key for your stream you want to play back. I first accidentally made a stupid mistake in not adding the Methodnames to its methods, but that's I think not the issue in my case now. 

When you now intend to play a content encrypted stream, according to apple, you have to open an AVUrlAsset, like:

 

aURL := TNSUrl.Wrap(TNSUrl.OCClass.URLWithString(StrToNSStr(url)));
FAsset := TAVURLAsset.Wrap(TAVURLAsset.OCClass.URLAssetWithURL(aURL, nil));

AVURLAsset has a property named hasProtectedContent which is true, when the stream is encrypted. In this case all you have to do is to provide recipient which provided the DRM key with a call named addContentKeyRecipient:

 

FContentKeySession.addContentKeyRecipient(FAsset);

The problem I have now is that when I provide my receiver asset to the AVContentKeySession the application terminates immediately. My work around, In order to get at least a system report, is to put the routine in a separate task via.

 

rocedure TfrmMain.FormActivate(Sender: TObject);
begin
  onactivate := nil;
  TTask.Create(
    procedure()
    var
      url: string;
      aURL: NSUrl;
    begin
      url := 'http://delphiworlds.s3-us-west-2.amazonaws.com/kastri-sponsor-video.mp4';
      aURL := TNSUrl.Wrap(TNSUrl.OCClass.URLWithString(StrToNSStr(url)));
      FAsset := TAVURLAsset.Wrap(TAVURLAsset.OCClass.URLAssetWithURL
        (aURL, nil));
      FAsset.retain;
      // if FAsset.hasProtectedContent then
      FContentKeySession.addContentKeyRecipient(FAsset);
    end).Start;
end;

The report then shows at least a bit information:

 

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

External Modification Warnings:
Debugger attached to process.

Application Specific Information:
dyld: in dlopen()
abort() called
terminating with uncaught foreign exception

Does someone has an idea what the reason for this behavior might be?

example.zip

[Dave Nottage 562]

1 hour ago, CHackbart said:

FContentKeySession.addContentKeyRecipient(FAsset)

Should be:

FContentKeySession.addContentKeyRecipient(NSObjectToID(FAsset))

You shouldn't need to use a TTask in the code there, either. I had issues compiling your test project, so I started a new one, and just added the form from the original project to it.

[CHackbart 13]

Wow, thank you very much. This seem to be the solution. I now receive the callbacks from the system. And it seem to work on MacOS as well as on iOS - except the fact that the AVFoundation API Headers (in iOS.AVFoundation) are missing, but luckily this is just copy and paste.

 

Again, thanks for the hint - this saved me a tremendous amount of peek in the poke.

 

Christian

Edited August 9, 2021 by CHackbart

[CHackbart 13]

I do have one additional question, you might be also able to answer.

 

Quote

let keyResponse = AVContentKeyResponse(fairPlayStreamingKeyResponseData: ckcData)

ckcData is NSData and seem to be type casted to AVContentKeyResponse.  I suppose something like keyResponse := TAVContentKeyResponse.Wrap(crcData) is not correct, right?

And how do I fill a NSDictionary like options: [AVContentKeyRequestProtocolVersionsKey: [1]], 

 

And there is one function makeStreamingContentKeyRequestDataForApp which wants a NSData which is technically a string assetIDData = assetIDString.data(using: .utf8)

 

Christian

Edited August 9, 2021 by CHackbart

[Dave Nottage 562]

2 hours ago, CHackbart said:

I suppose something like keyResponse := TAVContentKeyResponse.Wrap(crcData) is not correct, right?

It's not correct. It needs to be:

keyResponse := TAVContentKeyResponse.Wrap(TAVContentKeyResponse.OCClass.contentKeyResponseWithFairPlayStreamingKeyResponseData(ckcData));

 

2 hours ago, CHackbart said:

And how do I fill a NSDictionary like options: [AVContentKeyRequestProtocolVersionsKey: [1]], 

If it's an NSDictionary with only one object, this is an example using a typical pattern for that scenario:

dict := TNSDictionary.Wrap(TNSDictionary.OCClass.dictionaryWithObject(TNSNumber.OCClass.numberWithInt(1), NSObjectToID(AVContentKeyRequestProtocolVersionsKey));

When there's more than one value to add, one way is to create an instance of NSMutableDictionary, and use the setValue method. There's a couple of examples in FMX.AddressBook.iOS

2 hours ago, CHackbart said:

assetIDData = assetIDString.data(using: .utf8)

 

assetIDData := assetIDString.dataUsingEncoding(NSUTF8StringEncoding);

 

Edited August 9, 2021 by Dave Nottage

[CHackbart 13]

Thanks, the whole translation from this:

func requestApplicationCertificate() throws -> Data {
        
        print("requestApplicationCertificate called")
        // MARK: ADAPT - You must implement this method to retrieve your FPS application certificate.
        var certificateData: Data? = nil
        
        let request = NSMutableURLRequest(url: URL(string: DNSRestServices.DNS_FAIRLPLAY_SERVER_URL)!)
        request.httpMethod = "POST"
        request.cachePolicy = NSURLRequest.CachePolicy.reloadIgnoringCacheData
        request.setValue("application/octet-stream", forHTTPHeaderField: "Content-Type")
        
        let semaphore = DispatchSemaphore(value: 0)
        
        URLSession.shared.dataTask(with: request as URLRequest) { (responseData, _, error) -> Void in
            certificateData = responseData
            semaphore.signal()
            }.resume()
        
        semaphore.wait(timeout: .distantFuture)
        
        guard certificateData != nil else {
            throw ProgramError.missingApplicationCertificate
        }
        
        return certificateData!
    }

func requestContentKeyFromKeySecurityModule(spcData: Data, assetID: String) throws -> Data {
        
        var ckcData: Data? = nil
        
        var licenseURL : String?
        licenseURL = DispatchQueue.main.sync {
            var licenseURLForSelectedChannel = ""
            if let keyWindow = UIWindow.key {
                let menuViewController = keyWindow.rootViewController as! MenuSplitViewController
                let playerController = menuViewController.viewControllers[1] as! DNSPlayerViewController
                licenseURLForSelectedChannel = playerController.licenseURLForSelectedChannel() ?? ""
            }
            return licenseURLForSelectedChannel
        }
        
        
 
        guard licenseURL != nil else {
            throw ProgramError.missingLicenseURL
        }
        
        
        guard let url = URL(string: licenseURLString) else {
            print("Error! Invalid URL!") //Do something else
            throw ProgramError.missingLicenseURL
        }
        
        let request = NSMutableURLRequest(url: url)
        request.httpMethod = "POST"
        request.cachePolicy = NSURLRequest.CachePolicy.reloadIgnoringCacheData
        request.httpBody = spcData
        let postLength:NSString =  NSString(data: spcData, encoding:String.Encoding.ascii.rawValue)!
        request.setValue(String(postLength.length), forHTTPHeaderField: "Content-Length")
        request.setValue("application/octet-stream", forHTTPHeaderField: "Content-Type")
        
        let semaphore = DispatchSemaphore(value: 0)
        
        URLSession.shared.dataTask(with: request as URLRequest) { (responseData, _, error) -> Void in
            ckcData = responseData
            semaphore.signal()
            }.resume()
        
        semaphore.wait(timeout: .distantFuture)
        
        
        guard ckcData != nil else {
            throw ProgramError.noCKCReturnedByKSM
        }
        
        return ckcData!
    }

func handleStreamingContentKeyRequest(keyRequest: AVContentKeyRequest) {
        guard let contentKeyIdentifierString = keyRequest.identifier as? String,
            let contentKeyIdentifierURL = URL(string: contentKeyIdentifierString),
            let assetIDString = contentKeyIdentifierURL.host,
            let assetIDData = assetIDString.data(using: .utf8)
            else {
                print("Failed to retrieve the assetID from the keyRequest!")
                return
        }

        let provideOnlinekey: () -> Void = { () -> Void in

            do {
                let applicationCertificate = try self.requestApplicationCertificate()

                let completionHandler = { [weak self] (spcData: Data?, error: Error?) in
                    guard let strongSelf = self else { return }
                    if let error = error {
                        keyRequest.processContentKeyResponseError(error)
                        return
                    }

                    guard let spcData = spcData else { return }

                    do {
                        // Send SPC to Key Server and obtain CKC
                        let ckcData = try strongSelf.requestContentKeyFromKeySecurityModule(spcData: spcData, assetID: assetIDString)

                        /*
                         AVContentKeyResponse is used to represent the data returned from the key server when requesting a key for
                         decrypting content.
                         */
                        let keyResponse = AVContentKeyResponse(fairPlayStreamingKeyResponseData: ckcData)

                        /*
                         Provide the content key response to make protected content available for processing.
                         */
                        keyRequest.processContentKeyResponse(keyResponse)
                    } catch {
                        keyRequest.processContentKeyResponseError(error)
                    }
                }

                keyRequest.makeStreamingContentKeyRequestData(forApp: applicationCertificate,
                                                              contentIdentifier: assetIDData,
                                                              options: [AVContentKeyRequestProtocolVersionsKey: [1]],
                                                              completionHandler: completionHandler)
            } catch {
                keyRequest.processContentKeyResponseError(error)
            }
        }
        
        provideOnlinekey()
    }
}

 

should look more or less like this:

function TContentKeyDelegate.requestApplicationCertificate(): NSData;
var
  http: THTTPClient;
  Response: TMemoryStream;
begin
  http := THTTPClient.Create;
  Response := TMemoryStream.Create;
  try
    http.ContentType := 'application/octet-stream';
    http.Get(FAIRLPLAY_SERVER_URL, Response);
    result := TNSData.Wrap(TNSData.OCClass.dataWithBytes(response.Memory, response.Size))
  finally
    Response.Free;
    http.Free;
  end;
end;

function TContentKeyDelegate.requestContentKeyFromKeySecurityModule(spcData: NSData; assetID: NSString): NSData;
var
  http: THTTPClient;
  Response: TMemoryStream;
  Request: TMemoryStream;
begin
  http := THTTPClient.Create;
  Response := TMemoryStream.Create;
  Request := TMemoryStream.Create;
  Request.Write(spcData.bytes^, spcData.length);
  try
    Request.Position := 0;
    http.ContentType := 'application/octet-stream';
    http.Post(KEY_SERVER_URL, Request, Response);
    result := TNSData.Wrap(TNSData.OCClass.dataWithBytes(response.Memory, response.Size))
  finally
    Request.Free;
    Response.Free;
    http.Free;
  end;
end;

procedure TContentKeyDelegate.requestCompleteHandler(contentKeyRequestData: NSData;error: NSError);
var ckcData: NSData;
  keyResponse: AVContentKeyResponse;
begin
 ckcData := requestContentKeyFromKeySecurityModule(contentKeyRequestData, FAssetIDString);
 keyResponse := TAVContentKeyResponse.Wrap(TAVContentKeyResponse.OCClass.contentKeyResponseWithFairPlayStreamingKeyResponseData(ckcData));
 FKeyRequest.processContentKeyResponse(keyResponse);
end;

procedure TContentKeyDelegate.handleStreamingContentKeyRequest(keyRequest: AVContentKeyRequest);
var
  contentKeyIdentifierString: NSString;
  assetIDData: NSData;
  contentKeyIdentifierURL: NSURL;
  dictionary: NSDictionary;
begin
  FKeyRequest := keyRequest;
  contentKeyIdentifierString := TNSString.Wrap(keyRequest.identifier);
  contentKeyIdentifierURL := TNSUrl.Wrap(TNSUrl.OCClass.URLWithString(contentKeyIdentifierString));
  FAssetIDString := contentKeyIdentifierURL.host;
  assetIDData := FAssetIDString.dataUsingEncoding(NSUTF8Stringencoding);

  dictionary := TNSDictionary.Wrap(TNSDictionary.OCClass.dictionaryWithObject(TNSNumber.OCClass.numberWithInt(1), NSObjectToID(AVContentKeyRequestProtocolVersionsKey)));

  keyRequest.makeStreamingContentKeyRequestDataForApp(
    requestApplicationCertificate,
    assetIDData,
    dictionary,
    requestCompleteHandler);
end;

I can post the whole unit if somebody is interested in. Using it with the FMX.Media class is quite simple. All you need is to assign the asset with the ContentKeyManager:

 

LAsset := TAVURLAsset.Wrap(TAVURLAsset.OCClass.URLAssetWithURL(LURL, nil));  

 if LAsset.hasProtectedContent then ContentKeyManager.addContentKeyRecipient(LAsset);  

FPlayerItem := TAVPlayerItem.Wrap(TAVPlayerItem.OCClass.playerItemWithAsset(LAsset));   

 

Edited August 10, 2021 by CHackbart

[Dave Nottage 562]

13 hours ago, CHackbart said:

I can post the whole unit if somebody is interested in.

I'm interested, thanks!

[CHackbart 13]

Sure, 

 

sorry for the delay - I just forgot to post it here.

 

You have to update the FMX.Media.Mac resp. IOS in the following way:

 

constructor TMacMedia.Create(const AFileName: string);
var
  LURL: NSUrl;
  LAbsoluteFileName: string;
  LAsset: AVURLAsset;
begin
  inherited Create(AFileName);
  AVMediaTypeAudio; // Force load the framework
  if FileExists(FileName) then
  begin
    if ExtractFilePath(FileName).IsEmpty then
      LAbsoluteFileName := TPath.Combine(TPath.GetHomePath, FileName)
    else
      LAbsoluteFileName := FileName;
    LURL := TNSUrl.Wrap(TNSUrl.OCClass.fileURLWithPath(StrToNSStr(LAbsoluteFileName)));
  end
  else
    LURL := StrToNSUrl(FileName);
  if LURL = nil then
    raise EFileNotFoundException.Create(SSpecifiedFileNotFound);
  FPixelBufferBitmap := TBitmap.Create;
  LAsset := TAVURLAsset.Wrap(TAVURLAsset.OCClass.URLAssetWithURL(LURL, nil));
  if LAsset.hasProtectedContent then
   ContentKeyManager.addContentKeyRecipient(LAsset);

  FPlayerItem := TAVPlayerItem.Wrap(TAVPlayerItem.OCClass.playerItemWithAsset(LAsset));
  FPlayerItem.retain;
  FPlayer := TAVPlayer.Wrap(TAVPlayer.OCClass.playerWithPlayerItem(FPlayerItem));
  FPlayer.retain;
  FPlayerLayer := TAVPlayerLayer.Wrap(TAVPlayerLayer.OCClass.playerLayerWithPlayer(FPlayer));
  FPlayerLayer.retain;
  FPlayerLayer.setVideoGravity(CocoaNSStringConst(libAVFoundation, 'AVLayerVideoGravityResizeAspectFill'));
  FPlayerLayer.setAutoresizingMask(kCALayerWidthSizable or kCALayerHeightSizable);
  FVideoView := TNSView.Create;
  FVideoView.retain;
  FVideoView.setWantsLayer(True);
  FVideoView.layer.addSublayer(FPlayerLayer);
  SetupVideoOutput;
end;

The ContentKeyManager needs two callbacks which could look like this:

procedure TfrmMain.DoGetCertificate(Sender: TObject; ACert: TMemoryStream);
var
  http: THTTPClient;
begin
  http := THTTPClient.create;
  try
    http.ContentType := 'application/octet-stream';
    http.Get(FAIRLPLAY_SERVER_URL, ACert);
  finally
    http.Free;
  end;
end;

procedure TfrmMain.DoRequestContentKey(Sender: TObject;
  ARequest, AResponse: TMemoryStream);
var
  http: THTTPClient;
begin
  http := THTTPClient.create;
  try
    ARequest.Position := 0;
    http.ContentType := 'application/octet-stream';
    http.Post(LS_SERVER_URL, ARequest, AResponse);
  finally
    http.Free;
  end;
end;

First Fairplay asks for the certificate and returns a request to the license server which should be send as post command. In return it delivers some binary data which is the key to decode the stream.

 

I made an "example" screenshot with a test server here. The annoying thing is that DRM content does not allow to play the video in a texture. If you start to use copyPixelBufferForItemTime in order to get the video content audio and video playback stops (without any notification). I was thinking about how to put some alpha blended NSView or UIView on top which contain the buttons and other things. One of my projects involves a media player on Android, iOS and MacOS (using Metal). It heavily uses Metal and OpenGL for the output and the video view is drawn over it (using regular firemonkey controls). Maybe you have an idea? 

 

By default the video is shown on the top right and some ui is drawn above. In fullscreen the video is in the background and the rest of the context is shown in front. You can imagine what happen if DRM is involved. You only see the image, but the ui is hidden.

 

Christian

 

UFairplay.pas.zip

Edited August 18, 2021 by CHackbart