Jump to content


  • Content Count

  • Joined

  • Last visited

Community Reputation

6 Neutral

Recent Profile Visitors

The recent visitors block is disabled and is not being shown to other users.

  1. CHackbart

    Draw over a native element

    I am not quite happy with the solution, but it works on iOS and MacOS. Android is missing, during the lack of a working device here at the moment, maybe somebody would like to add the Android native Image handling into. The idea is to have a TLayout descendant which renders the Firemonkey content into a bitmap and draws this to a native Image View. What I have so far is the following: unit UImageLayout; interface uses System.Types, System.Classes, FMX.Layouts, FMX.Types {$IFDEF MACOS} {$IFDEF IOS} , iOSApi.UIKit {$ELSE} , Macapi.AppKit {$ENDIF} {$ENDIF}; type TImageLayout = class(TLayout) private FActive: Boolean; FLastUpdate: UInt64; {$IFDEF MACOS} {$IFDEF IOS} FView: UIImageView; {$ELSE} FView: NSImageView; {$ENDIF} {$ENDIF} procedure SetActive(AValue: Boolean); protected procedure AfterPaint; override; public constructor Create(AOwner: TComponent); override; destructor Destroy; override; property Active: Boolean read FActive write SetActive; end; implementation uses System.SysUtils, FMX.Graphics, FMX.Forms, System.IOUtils {$IFDEF MACOS} {$IFDEF IOS} , Macapi.Helpers, iOSApi.Foundation, FMX.Platform.iOS {$ELSE} , Macapi.CocoaTypes, FMX.Platform.Mac, FMX.Helpers.Mac {$ENDIF} {$ENDIF}; {$IFDEF MACOS} {$IFDEF IOS} function BitmapToUIImage(const Bitmap: TBitmap): UIImage; var mem: TMemoryStream; Data: NSData; TmpImage: UIImage; AutoreleasePool: NSAutoreleasePool; begin if Bitmap <> nil then begin Data := nil; TmpImage := nil; AutoreleasePool := TNSAutoreleasePool.Create; try mem := TMemoryStream.Create; try Bitmap.SaveToStream(mem); Data := TNSData.Wrap(TNSData.OCClass.dataWithBytes(mem.Memory, mem.size)); TmpImage := TUIImage.Alloc; Result := TUIImage.Wrap(TmpImage.initWithData(Data)); finally mem.Free; end; finally AutoreleasePool.release; end; end; end; {$ENDIF} function MakeNSRect(const ox, oy, sx, sy: Single): NSRect; begin Result.origin.x := ox; Result.origin.y := oy; Result.size.width := sx; Result.size.height := sy; end; {$ENDIF} function GetParentForm(Control: TFmxObject): TCommonCustomForm; begin if (Control.Root <> nil) and (Control.Root.GetObject is TCommonCustomForm) then Result := TCommonCustomForm(Control.Root.GetObject) else Result := nil; end; { TImageLayout } constructor TImageLayout.Create(AOwner: TComponent); begin inherited; end; destructor TImageLayout.Destroy; begin Active := false; inherited; end; procedure TImageLayout.AfterPaint; var LBitmap: TBitmap; {$IFDEF MACOS} LBounds: TRectF; Form: TCommonCustomForm; {$ENDIF} begin inherited; if (not Active) then exit; if TThread.GetTickCount64 - FLastUpdate < 100 then exit; FLastUpdate := TThread.GetTickCount64; {$IFDEF MACOS} if assigned(FView) then begin LBounds := GetAbsoluteRect; Form := GetParentForm(self); if assigned(Form) then FView.setFrame(MakeNSRect(LBounds.Left, Form.ClientHeight - LBounds.Bottom, LBounds.width, LBounds.height)); end; {$ENDIF} LBitmap := TBitmap.Create(Trunc(width), Trunc(height)); try if LBitmap.Canvas.BeginScene then begin LBitmap.Canvas.Clear(0); PaintTo(LBitmap.Canvas, LBitmap.BoundsF); LBitmap.Canvas.EndScene; //LBitmap.SaveToFile(TPath.Combine(TPath.GetDocumentsPath, 'test.png')); {$IFDEF MACOS} {$IFDEF IOS} FView.setAlpha(AbsoluteOpacity); FView.setImage(BitmapToUIImage(LBitmap)); {$ELSE} FView.setAlphaValue(AbsoluteOpacity); FView.setImage(BitmapToMacBitmap(LBitmap)); {$ENDIF} {$ENDIF} end; finally LBitmap.Free; end; end; procedure TImageLayout.SetActive(AValue: Boolean); {$IFDEF MACOS} var LBounds: TRectF; Form: TCommonCustomForm; {$IFDEF IOS} FormView: UIView; {$ELSE} FormView: NSView; {$ENDIF} {$ENDIF} begin if AValue <> Active then begin {$IFDEF MACOS} LBounds := AbsoluteRect; Form := GetParentForm(self); if (AValue) and (not assigned(FView)) and (assigned(Form)) then begin {$IFDEF IOS} FView := TUIImageView.Wrap(TUIImageView.Alloc.initWithFrame (MakeNSRect(LBounds.Left, Form.ClientHeight - LBounds.Bottom, LBounds.width, LBounds.height))); FView.retain; FormView := TUIView.Wrap (NSObjectToID(WindowHandleToPlatform(Form.Handle).View)); FormView.addSubview(FView); {$ELSE} FView := TNSImageView.Wrap(TNSImageView.Alloc.initWithFrame (MakeNSRect(LBounds.Left, Form.ClientHeight - LBounds.Bottom, LBounds.width, LBounds.height))); FView.retain; FView.setWantsLayer(true); FormView := TNSView.Wrap(WindowHandleToPlatform(Form.Handle) .Wnd.contentView); FormView.addSubview(FView, NSWindowAbove, nil); {$ENDIF} end; if assigned(FView) then FView.setHidden(not AValue) else AValue := false; {$ENDIF} FActive := AValue; end; end; end.
  2. CHackbart

    Draw over a native element

    Hi, i am wondering if there is an elegant way to draw over a native control like the Media player or Webbrowser under mobile platforms resp. MacOS. In my case I have to use AVPlayerItemVideoOutput because the content is DRM protected and I can not get the surface texture in order to paint it to the screen, embedded in Firemonkey controls. My solution is to render the FireMonkey output into a bitmap and to draw it on a NSImageView Layer which is on top. This is slow as hell and I wonder how it can be achieved in a better way than using an image. Christian
  3. CHackbart

    MacOS AVPlayer and DRM

    Sure, sorry for the delay - I just forgot to post it here. You have to update the FMX.Media.Mac resp. IOS in the following way: constructor TMacMedia.Create(const AFileName: string); var LURL: NSUrl; LAbsoluteFileName: string; LAsset: AVURLAsset; begin inherited Create(AFileName); AVMediaTypeAudio; // Force load the framework if FileExists(FileName) then begin if ExtractFilePath(FileName).IsEmpty then LAbsoluteFileName := TPath.Combine(TPath.GetHomePath, FileName) else LAbsoluteFileName := FileName; LURL := TNSUrl.Wrap(TNSUrl.OCClass.fileURLWithPath(StrToNSStr(LAbsoluteFileName))); end else LURL := StrToNSUrl(FileName); if LURL = nil then raise EFileNotFoundException.Create(SSpecifiedFileNotFound); FPixelBufferBitmap := TBitmap.Create; LAsset := TAVURLAsset.Wrap(TAVURLAsset.OCClass.URLAssetWithURL(LURL, nil)); if LAsset.hasProtectedContent then ContentKeyManager.addContentKeyRecipient(LAsset); FPlayerItem := TAVPlayerItem.Wrap(TAVPlayerItem.OCClass.playerItemWithAsset(LAsset)); FPlayerItem.retain; FPlayer := TAVPlayer.Wrap(TAVPlayer.OCClass.playerWithPlayerItem(FPlayerItem)); FPlayer.retain; FPlayerLayer := TAVPlayerLayer.Wrap(TAVPlayerLayer.OCClass.playerLayerWithPlayer(FPlayer)); FPlayerLayer.retain; FPlayerLayer.setVideoGravity(CocoaNSStringConst(libAVFoundation, 'AVLayerVideoGravityResizeAspectFill')); FPlayerLayer.setAutoresizingMask(kCALayerWidthSizable or kCALayerHeightSizable); FVideoView := TNSView.Create; FVideoView.retain; FVideoView.setWantsLayer(True); FVideoView.layer.addSublayer(FPlayerLayer); SetupVideoOutput; end; The ContentKeyManager needs two callbacks which could look like this: procedure TfrmMain.DoGetCertificate(Sender: TObject; ACert: TMemoryStream); var http: THTTPClient; begin http := THTTPClient.create; try http.ContentType := 'application/octet-stream'; http.Get(FAIRLPLAY_SERVER_URL, ACert); finally http.Free; end; end; procedure TfrmMain.DoRequestContentKey(Sender: TObject; ARequest, AResponse: TMemoryStream); var http: THTTPClient; begin http := THTTPClient.create; try ARequest.Position := 0; http.ContentType := 'application/octet-stream'; http.Post(LS_SERVER_URL, ARequest, AResponse); finally http.Free; end; end; First Fairplay asks for the certificate and returns a request to the license server which should be send as post command. In return it delivers some binary data which is the key to decode the stream. I made an "example" screenshot with a test server here. The annoying thing is that DRM content does not allow to play the video in a texture. If you start to use copyPixelBufferForItemTime in order to get the video content audio and video playback stops (without any notification). I was thinking about how to put some alpha blended NSView or UIView on top which contain the buttons and other things. One of my projects involves a media player on Android, iOS and MacOS (using Metal). It heavily uses Metal and OpenGL for the output and the video view is drawn over it (using regular firemonkey controls). Maybe you have an idea? By default the video is shown on the top right and some ui is drawn above. In fullscreen the video is in the background and the rest of the context is shown in front. You can imagine what happen if DRM is involved. You only see the image, but the ui is hidden. Christian UFairplay.pas.zip
  4. CHackbart

    MacOS AVPlayer and DRM

    Thanks, the whole translation from this: func requestApplicationCertificate() throws -> Data { print("requestApplicationCertificate called") // MARK: ADAPT - You must implement this method to retrieve your FPS application certificate. var certificateData: Data? = nil let request = NSMutableURLRequest(url: URL(string: DNSRestServices.DNS_FAIRLPLAY_SERVER_URL)!) request.httpMethod = "POST" request.cachePolicy = NSURLRequest.CachePolicy.reloadIgnoringCacheData request.setValue("application/octet-stream", forHTTPHeaderField: "Content-Type") let semaphore = DispatchSemaphore(value: 0) URLSession.shared.dataTask(with: request as URLRequest) { (responseData, _, error) -> Void in certificateData = responseData semaphore.signal() }.resume() semaphore.wait(timeout: .distantFuture) guard certificateData != nil else { throw ProgramError.missingApplicationCertificate } return certificateData! } func requestContentKeyFromKeySecurityModule(spcData: Data, assetID: String) throws -> Data { var ckcData: Data? = nil var licenseURL : String? licenseURL = DispatchQueue.main.sync { var licenseURLForSelectedChannel = "" if let keyWindow = UIWindow.key { let menuViewController = keyWindow.rootViewController as! MenuSplitViewController let playerController = menuViewController.viewControllers[1] as! DNSPlayerViewController licenseURLForSelectedChannel = playerController.licenseURLForSelectedChannel() ?? "" } return licenseURLForSelectedChannel } guard licenseURL != nil else { throw ProgramError.missingLicenseURL } guard let url = URL(string: licenseURLString) else { print("Error! Invalid URL!") //Do something else throw ProgramError.missingLicenseURL } let request = NSMutableURLRequest(url: url) request.httpMethod = "POST" request.cachePolicy = NSURLRequest.CachePolicy.reloadIgnoringCacheData request.httpBody = spcData let postLength:NSString = NSString(data: spcData, encoding:String.Encoding.ascii.rawValue)! request.setValue(String(postLength.length), forHTTPHeaderField: "Content-Length") request.setValue("application/octet-stream", forHTTPHeaderField: "Content-Type") let semaphore = DispatchSemaphore(value: 0) URLSession.shared.dataTask(with: request as URLRequest) { (responseData, _, error) -> Void in ckcData = responseData semaphore.signal() }.resume() semaphore.wait(timeout: .distantFuture) guard ckcData != nil else { throw ProgramError.noCKCReturnedByKSM } return ckcData! } func handleStreamingContentKeyRequest(keyRequest: AVContentKeyRequest) { guard let contentKeyIdentifierString = keyRequest.identifier as? String, let contentKeyIdentifierURL = URL(string: contentKeyIdentifierString), let assetIDString = contentKeyIdentifierURL.host, let assetIDData = assetIDString.data(using: .utf8) else { print("Failed to retrieve the assetID from the keyRequest!") return } let provideOnlinekey: () -> Void = { () -> Void in do { let applicationCertificate = try self.requestApplicationCertificate() let completionHandler = { [weak self] (spcData: Data?, error: Error?) in guard let strongSelf = self else { return } if let error = error { keyRequest.processContentKeyResponseError(error) return } guard let spcData = spcData else { return } do { // Send SPC to Key Server and obtain CKC let ckcData = try strongSelf.requestContentKeyFromKeySecurityModule(spcData: spcData, assetID: assetIDString) /* AVContentKeyResponse is used to represent the data returned from the key server when requesting a key for decrypting content. */ let keyResponse = AVContentKeyResponse(fairPlayStreamingKeyResponseData: ckcData) /* Provide the content key response to make protected content available for processing. */ keyRequest.processContentKeyResponse(keyResponse) } catch { keyRequest.processContentKeyResponseError(error) } } keyRequest.makeStreamingContentKeyRequestData(forApp: applicationCertificate, contentIdentifier: assetIDData, options: [AVContentKeyRequestProtocolVersionsKey: [1]], completionHandler: completionHandler) } catch { keyRequest.processContentKeyResponseError(error) } } provideOnlinekey() } } should look more or less like this: function TContentKeyDelegate.requestApplicationCertificate(): NSData; var http: THTTPClient; Response: TMemoryStream; begin http := THTTPClient.Create; Response := TMemoryStream.Create; try http.ContentType := 'application/octet-stream'; http.Get(FAIRLPLAY_SERVER_URL, Response); result := TNSData.Wrap(TNSData.OCClass.dataWithBytes(response.Memory, response.Size)) finally Response.Free; http.Free; end; end; function TContentKeyDelegate.requestContentKeyFromKeySecurityModule(spcData: NSData; assetID: NSString): NSData; var http: THTTPClient; Response: TMemoryStream; Request: TMemoryStream; begin http := THTTPClient.Create; Response := TMemoryStream.Create; Request := TMemoryStream.Create; Request.Write(spcData.bytes^, spcData.length); try Request.Position := 0; http.ContentType := 'application/octet-stream'; http.Post(KEY_SERVER_URL, Request, Response); result := TNSData.Wrap(TNSData.OCClass.dataWithBytes(response.Memory, response.Size)) finally Request.Free; Response.Free; http.Free; end; end; procedure TContentKeyDelegate.requestCompleteHandler(contentKeyRequestData: NSData;error: NSError); var ckcData: NSData; keyResponse: AVContentKeyResponse; begin ckcData := requestContentKeyFromKeySecurityModule(contentKeyRequestData, FAssetIDString); keyResponse := TAVContentKeyResponse.Wrap(TAVContentKeyResponse.OCClass.contentKeyResponseWithFairPlayStreamingKeyResponseData(ckcData)); FKeyRequest.processContentKeyResponse(keyResponse); end; procedure TContentKeyDelegate.handleStreamingContentKeyRequest(keyRequest: AVContentKeyRequest); var contentKeyIdentifierString: NSString; assetIDData: NSData; contentKeyIdentifierURL: NSURL; dictionary: NSDictionary; begin FKeyRequest := keyRequest; contentKeyIdentifierString := TNSString.Wrap(keyRequest.identifier); contentKeyIdentifierURL := TNSUrl.Wrap(TNSUrl.OCClass.URLWithString(contentKeyIdentifierString)); FAssetIDString := contentKeyIdentifierURL.host; assetIDData := FAssetIDString.dataUsingEncoding(NSUTF8Stringencoding); dictionary := TNSDictionary.Wrap(TNSDictionary.OCClass.dictionaryWithObject(TNSNumber.OCClass.numberWithInt(1), NSObjectToID(AVContentKeyRequestProtocolVersionsKey))); keyRequest.makeStreamingContentKeyRequestDataForApp( requestApplicationCertificate, assetIDData, dictionary, requestCompleteHandler); end; I can post the whole unit if somebody is interested in. Using it with the FMX.Media class is quite simple. All you need is to assign the asset with the ContentKeyManager: LAsset := TAVURLAsset.Wrap(TAVURLAsset.OCClass.URLAssetWithURL(LURL, nil)); if LAsset.hasProtectedContent then ContentKeyManager.addContentKeyRecipient(LAsset); FPlayerItem := TAVPlayerItem.Wrap(TAVPlayerItem.OCClass.playerItemWithAsset(LAsset));
  5. CHackbart

    MacOS AVPlayer and DRM

    I do have one additional question, you might be also able to answer. ckcData is NSData and seem to be type casted to AVContentKeyResponse. I suppose something like keyResponse := TAVContentKeyResponse.Wrap(crcData) is not correct, right? And how do I fill a NSDictionary like options: [AVContentKeyRequestProtocolVersionsKey: [1]], And there is one function makeStreamingContentKeyRequestDataForApp which wants a NSData which is technically a string assetIDData = assetIDString.data(using: .utf8) Christian
  6. CHackbart

    MacOS AVPlayer and DRM

    Wow, thank you very much. This seem to be the solution. I now receive the callbacks from the system. And it seem to work on MacOS as well as on iOS - except the fact that the AVFoundation API Headers (in iOS.AVFoundation) are missing, but luckily this is just copy and paste. Again, thanks for the hint - this saved me a tremendous amount of peek in the poke. Christian
  7. CHackbart

    MacOS AVPlayer and DRM

    Hi, I'm not sure if I did something wrong, but it is nearly impossible to debug - since the application is crashing without any further error message after providing the necessary information to the AVContentKeySession. Apples documentation is also quite rudimentary when it comes to proper handling of DRM content. In theory you create a content key session: FContentKeySession := TAVContentKeySession.Wrap(TAVContentKeySession.OCClass.contentKeySessionWithKeySystem (AVContentKeySystemFairPlayStreaming)); // FairPlayStreaming Next you implement your own implementation of AVContentKeySessionDelegate which handles receiving the certificate and key for your stream you want to play back. I first accidentally made a stupid mistake in not adding the Methodnames to its methods, but that's I think not the issue in my case now. When you now intend to play a content encrypted stream, according to apple, you have to open an AVUrlAsset, like: aURL := TNSUrl.Wrap(TNSUrl.OCClass.URLWithString(StrToNSStr(url))); FAsset := TAVURLAsset.Wrap(TAVURLAsset.OCClass.URLAssetWithURL(aURL, nil)); AVURLAsset has a property named hasProtectedContent which is true, when the stream is encrypted. In this case all you have to do is to provide recipient which provided the DRM key with a call named addContentKeyRecipient: FContentKeySession.addContentKeyRecipient(FAsset); The problem I have now is that when I provide my receiver asset to the AVContentKeySession the application terminates immediately. My work around, In order to get at least a system report, is to put the routine in a separate task via. rocedure TfrmMain.FormActivate(Sender: TObject); begin onactivate := nil; TTask.Create( procedure() var url: string; aURL: NSUrl; begin url := 'http://delphiworlds.s3-us-west-2.amazonaws.com/kastri-sponsor-video.mp4'; aURL := TNSUrl.Wrap(TNSUrl.OCClass.URLWithString(StrToNSStr(url))); FAsset := TAVURLAsset.Wrap(TAVURLAsset.OCClass.URLAssetWithURL (aURL, nil)); FAsset.retain; // if FAsset.hasProtectedContent then FContentKeySession.addContentKeyRecipient(FAsset); end).Start; end; The report then shows at least a bit information: Crashed Thread: 0 Dispatch queue: com.apple.main-thread Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Exception Note: EXC_CORPSE_NOTIFY External Modification Warnings: Debugger attached to process. Application Specific Information: dyld: in dlopen() abort() called terminating with uncaught foreign exception Does someone has an idea what the reason for this behavior might be? example.zip