Pierre le Riche

Haha, I am with you there 100%. It is poorly documented and the error messages are not helpful at all. I spend hours getting a few lines of code working... it really makes me feel unproductive. Thank you for all your help so far. I think I'm starting to see the light at the end of the tunnel. I'll post my code here once I get it working.

Thank you, that import via pipe code will definitely come in handy as well. I've currently got all imports and exports between formats going via temporary disk file, which is not ideal. My next task is going to be to be able to intercept when the user clicks the save button and then store the updated document in the database. I still need to figure out how to intercept the click of the button so I know when the user has attempted to save the document (I will search for some examples online), but I think using a pipe might solve both the actual loading and saving. I did a quick test specifying a SequenceOutputStream as "OutputStream" in the arguments when loading the document, and then when the user clicks the save button it works the first time, but raises an exception on the second. (The error on the second save attempt is not unexpected, given that I didn't retrieve the data of the first save.)

Just an update on this. It turns out it's not an OLE Automation compatibility issue between Delphi and LibreOffice as I was starting to fear. Apparently the inner array that you pass to the initialize method of SequenceInputStream must be a "strongly typed uno value", so you have to massage the input a little bit. For the benefit of anyone stumbling upon this thread in future, here is how you load a LibreOffice document from a memory buffer: function LoadLibreOfficeDocumentFromMemory(const ADocumentFileData: TBytes): Variant; var LServiceManager, LStrictlyTypedUnoValue, LInputStream, LDesktop: Variant; LArgumentsArray, LPropertyArray: TArray<Variant>; begin LServiceManager := CreateOleObject('com.sun.star.ServiceManager'); {A SequenceInputStream is used to load the document from memory. In order to get the bytes into the SequenceInputStream the initialize method is called. This method takes an array of arguments, in which it expects to find a single entry which must be a strictly typed uno value - a byte array.} {Create the argument array for SequenceInputStream.initialize} LStrictlyTypedUnoValue := LServiceManager.Bridge_GetValueObject; LStrictlyTypedUnoValue.Set('[]byte', Variant(ADocumentFileData)); SetLength(LArgumentsArray, 1); LArgumentsArray[0] := LStrictlyTypedUnoValue; {Create and initialize the SequenceInputStream} LInputStream := LServiceManager.createInstance('com.sun.star.io.SequenceInputStream'); LInputStream.initialize(Variant(LArgumentsArray)); {Specify the stream to load from in the property array for loadComponentFromURL.} SetLength(LPropertyArray, 1); LPropertyArray[0] := LServiceManager.Bridge_GetStruct('com.sun.star.beans.PropertyValue'); LPropertyArray[0].Name := 'InputStream'; LPropertyArray[0].Value := LInputStream; {Load and display the document} LDesktop := LServiceManager.createInstance('com.sun.star.frame.Desktop'); Result := LDesktop.loadComponentFromURL('private:stream', '_blank', 0, Variant(LPropertyArray)); end;

Thanks, I sent the author an e-mail earlier this week, but he hasn't responded yet. I got loading from disk working, but loading from a memory buffer is turning out to be quite tricky. I took a look at the demo application for "Libre", but there was no example for loading from memory. I'll download the trial and poke around in the list of calls to see if it is able to do it.

Thanks for looking into this. When I replace SequenceInputStream with SequenceOutputStream in the attached example then the loadComponentFromURL crashes with this error: Project LibreOfficeLoadingFromStream.exe raised exception class EOleException with message 'com.sun.star.lang.IllegalArgumentException: Unsupported URL <private:stream>: "from LoadEnv::startLoading"'. It appears that it doesn't like a SequenceOutputStream specified for the InputStream property. If I use a SequenceInputStream and I skip the step of populating the stream then it gets to the point of loading the document before it complains that it cannot seek in the stream, which is not surprising given that the stream is empty. The documentation for the media description used by loadComponentFromURL (https://api.libreoffice.org/docs/idl/ref/servicecom_1_1sun_1_1star_1_1document_1_1MediaDescriptor.html#a7d6a9c7512c158efd9e12a78b1258be4) states that SequenceInputStream is for loading documents and SequenceOutputStream is for saving.

I am battling with LibreOffice integration. I have managed to get the loading of a writer document from disk working, but for the life of me I cannot figure out how to load a document from in-memory data. A small test case is attached with the line on which it crashes indicated in the source. You just need LibreOffice installed and the test app should run. The line on which it crashes is where I am trying to get the in-memory data into the stream from which to load the document. If I comment out that line then it runs through to the end (where it complains that the stream contains no data, as expected). The Delphi code is a loose translation of the Python code I found here, circa line 163: https://github.com/unoconv/unoserver/blob/master/src/unoserver/comparer.py I also found this seemingly similar question on the LibreOffice forums, but I had no success following their suggestions: https://ask.libreoffice.org/t/macro-basic-sequenceinputstream-createstreamfromsequence-runtime-error-not-found/110235 Anyone know what I am doing wrong? I have tried every permutation of input to SequenceInputStream.initialize that I could think of without success. Thanks! LibreOfficeLoadingFromStream.dpr

Thanks. I have already implemented the notification system using LdrRegisterDllNotification as suggested by Anders, and from user feedback it seems to prevent the annoying A/Vs. I'm goint to test how your suggestion performs, but I reckon it would be hard to beat caching of an address space map vs walking the list of modules before every stack trace. Currently I only map the low 4GB of address space, and assume that no DLL will be loaded above that. It's not an accurate assumption, but it seems to hold where it matters. If it becomes an issue I will definitely have to revisit this.

Thanks Anders, I think I'll go with the LdrRegisterDllNotification option. The Vista+ restriction shouldn't be a big deal, since I only need to deal with the exceptions under a debugger and I can't imagine too many developers are still using Windows XP. I see there's a note "[This function may be changed or removed from Windows without further notice.]" in the documentation, so I'll implement it in such a way that it doesn't crash if it it is not available. Best regards, Pierre

Hi, For performance reasons I am caching a map of the address space in the stack tracing code inside FastMM_FullDebugMode.dll. This map goes stale whenever a library is loaded or unloaded, and I've been dealing with this rather crudely: by handling the ensuing access violations in a try...except block. These access violations can be quite annoying when running the application under the debugger, so I would like to make them less likely. Invalidating the map whenever a library is loaded or unloaded would be an improvement. I've therefore been looking at ways to be notified when a library is loaded or unloaded, but the only solution I have found is patching the LoadLibrary and FreeLibrary calls in kernel32.dll in-memory to insert a hook. Is there a better strategy? The mechanism doesn't have to be 100% accurate, so if there's a proxy for LoadLibrary and FreeLibrary that is less invasive I would rather go for that. Thanks, Pierre

If the call was made by a DLL that was unloaded prior to the report being generated it won't be possible to convert the return address to call address after the fact. But I take your point - in most cases it'll work. If it's a big issue for you I can put it on my to-do list. TJclStackInfoList.ValidCallSite does that - it returns the size of the call instruction given a return address.

I agree with you, but calculating the call address is only included in the cost when you perform a raw stack trace: Frame based traces (including the CaptureStackBackTrace API call) yield return addresses, so in order to get the call addresses for those you would need to dereference the return pointers (as you do with raw traces) in order to find the start of the call instruction. This would negate much of the performance advantage over raw traces. Before a stack trace address is passed to the JCL for conversion to unit and line number information, 1 is subtracted in order to ensure the address falls inside the call instruction. Consequently the unit/line information corresponds to the call, and the listed address is the return address. Not ideal, but I find that I very rarely actually look at the address so it has never really bothered me.

Hi Stefan, The code that validates stack trace entries in FastMM_FullDebugMode.dpr was taken from TJclStackInfoList.ValidCallSite, with some minor modifications. (Credit is given in the comments.) I just checked and it is still the same, functionally, as the latest code in JCLDebug.pas. By default FastMM does a "raw" stack trace, meaning it just walks the stack in reverse, testing every dword to see whether it could potentially be the return address for a call instruction. It does this by checking whether the bytes just prior to the potential return address match one of the many potential opcodes for a call instruction. This is not 100% accurate, so you do get some false positives (as you have seen), but it works fairly well in general. The alternative would be to do a frame based stack trace, but then routines that do not set up a stack frame would not be listed at all. There's room for optimization here: The stack tracer could perhaps look at the actual address of the call instruction and compare that to the return address of the next call in the call stack. If the variance in code addresses is too large then the prior call is likely a false positive. Perhaps MadExcept does something like this, or some other smart tricks I am not aware of. I'll look around to see if there are ways to cut down on the false positives without impacting performance too much. I ran your test case under Delphi 10.4.2 and I got this stack trace: 0041320A [FastMM5.pas][FastMM5][FastMM_DebugGetMem$qqri][7717] 00404799 [System.pas][System][@ReallocMem$qqrrpvi][5035] 00408B4F [System.pas][System][DynArraySetLength$qqrrpvpvipi][36568] 00408CB6 [System.pas][System][@DynArraySetLength$qqrv][36672] 00427619 76F5FA29 [BaseThreadInitThunk] 77247A4E [RtlGetAppContainerNamedObjectPath] 77247A1E [RtlGetAppContainerNamedObjectPath] As you can see the call to LStrFromPWCharLen isn't there, so I think this confirms that it was just "noise" in the stack trace. Best regards, Pierre

In the SetLength call you're calling Integer.ToString and BoolToStr in order to determine the resultant string lengths, and lower down you're calling those again to get the actual strings. I reckon that's where the 30% is going. Even if you fix that I doubt you'll see more than a marginal performance improvement going from PrepareLineForExport and PrepareLineForExport_MOVE. String concatenation is decently implemented in the RTL. (The 32-bit implementation is even in assembly language.)

If you have applications that are currently using borlndmm.dll you can just replace it with one compiled using FastMM5, The source for it is in the "BorlndMM DLL" subfolder. For new applications I recommend that you use FastMM5 directly - add it as the first unit in your project's dpr file.

I have now added support for this. Previously you needed to execute "Include(FastMM_MessageBoxEvents, mmetUnexpectedMemoryLeakSummary);" to get a leak summary on shutdown, but now if ReportMemoryLeaksOnShutdown = True it will do that automatically. FastMM5 does not support the v4 options file, but it does support the v4 defines if you declare then in Project - Options - Delphi Compiler - Conditionals defines. If that is not convenient I recommend you use the equivalent v5 options instead - the v4 conditional defines support is just for backward compatibility.