Rtti multi-thread deadlock

[havrlisan 3]

Hi. I ran across a deadlock situation in the System.Rtti on the OS X platform. Below is the call stack for both threads that lock each other:

Thread 1:
Fmx::Forms::TFrame::TFrame(System::Classes::TComponent*) + 299 (FMX.Forms.pas:7524,1 in SampleApp + 8570027) [0x1017c54ab] 1-25
System::Classes::InitInheritedComponent(System::Classes::TComponent*, System::TMetaClass*) + 107 (System.Classes.pas:4859,1 in SampleApp + 1097787) [0x1010a503b] 1-25
System::Classes::BeginGlobalLoading() + 13 (System.Classes.pas:4805,1 in SampleApp + 1096637) [0x1010a4bbd] 1-25
System::Rtti::TRttiContext::KeepContext() + 123 (System.Rtti.pas:5665,1 in SampleApp + 1462715) [0x1010fe1bb] 1-25
System::Rtti::EnsurePoolToken(System::DelphiInterface<System::IInterface>*) + 55 (System.Rtti.pas:5319,1 in SampleApp + 1607015) [0x101121567] 1-25
System::Rtti::EnsurePoolToken(System::DelphiInterface<System::IInterface>*)::DoCreate(void*) + 72 (System.Rtti.pas:5306,1 in SampleApp + 1606392) [0x1011212f8] 1-25
System::Rtti::TPoolToken::TPoolToken() + 197 (System.Rtti.pas:5253,1 in SampleApp + 1605765) [0x101121085] 1-25
System::TMonitor::Enter
  
Thread 2:
System::Rtti::TRttiInstanceMethodEx::GetAttributes() + 41 (System.Rtti.pas:6452,1 in SampleApp + 1637625) [0x101128cf9] 1-25
System::Rtti::TRttiObject::GetAttributes() + 86 (System.Rtti.pas:5788,1 in SampleApp + 1415254) [0x1010f2856] 1-25
__stub_in48s__ZN6System4Rtti37LazyLoadAttributes_MakeClosure_ActRec7_0_BodyEv + 27 (SampleApp + 1622027) [0x10112500b] 1-25
System::Rtti::LazyLoadAttributes_MakeClosure_ActRec::_0_Body() + 591 (System.Rtti.pas:5537,1 in SampleApp + 1622639) [0x10112526f] 1-25
System::Rtti::ConstructAttributes(unsigned char*) + 176 (System.Rtti.pas:5496,1 in SampleApp + 1612512) [0x101122ae0] 1-25
System::Rtti::ConstructAttributes(unsigned char*)::ConstructAttribute(void*, unsigned char*&) + 631 (System.Rtti.pas:5476,1 in SampleApp + 1611943) [0x1011228a7] 1-25
System::Rtti::TRttiMethod::Invoke(System::TMetaClass*, System::Rtti::TValue const*, long long) + 234 (System.Rtti.pas:10149,1 in SampleApp + 1437914) [0x1010f80da] 1-25
System::Rtti::TRttiInstanceMethodEx::DispatchInvoke(System::Rtti::TValue const&, System::Rtti::TValue const*, long long) + 3057 (System.Rtti.pas:6620,1 in SampleApp + 1641665) [0x101129cc1] 1-25
System::Rtti::Invoke(void*, System::DynamicArray<System::Rtti::TValue>, System::Typinfo::TCallConv, System::Typinfo::TTypeInfo*, bool, bool) + 183 (System.Rtti.pas:9346,1 in SampleApp + 1634871) [0x101128237] 1-25
System::Rtti::TRttiContext::Create() + 28 (System.Rtti.pas:5577,1 in SampleApp + 1462540) [0x1010fe10c] 1-25
System::Rtti::EnsurePoolToken(System::DelphiInterface<System::IInterface>*) + 55 (System.Rtti.pas:5319,1 in SampleApp + 1607015) [0x101121567] 1-25
System::Rtti::EnsurePoolToken(System::DelphiInterface<System::IInterface>*)::DoCreate(void*) + 29 (System.Rtti.pas:5304,1 in SampleApp + 1606349) [0x1011212cd] 1-25
System::Rtti::TRttiContext::UseContext() + 119 (System.Rtti.pas:5687,1 in SampleApp + 1606663) [0x101121407] 1-25
System::TMonitor::Enter(unsigned int) + 526 (System.pas:20052,1 in SampleApp + 131422) [0x100fb915e] 1-25

 

The main issue is the order of entering the TMonitor locks, which I extracted here:

Thread 1:
TMonitor.Enter(GCTokenLock);       -- 5663:TRttiContext.KeepContext
TMonitor.Enter(PoolLock);          -- 5253:TPoolToken.Create

Thread 2:
TMonitor.Enter(PoolLock);          -- 5524:LazyLoadAttributes
TMonitor.Enter(GCTokenLock);       -- 5687:TRttiContext.UseContext

 

Am I at fault for reproducing these two call stacks in multiple threads (creating a frame in one, calling TRttiMethod.GetAttributes in the other), or is this a bug in the System.Rtti unit?

 

It's worth noting that the GCTokenLock object (and locking) is only present with the USE_MONITOR_FOR_GLOBALCONTEXT compiler directive, otherwise atomic operations are used:

{$IF Defined(WIN32) or Defined(WIN64)}
  {$UNDEF USE_MONITOR_FOR_GLOBALCONTEXT}
{$ELSE}
  {$DEFINE USE_MONITOR_FOR_GLOBALCONTEXT}
{$ENDIF}

 

[Vincent Parrett 657]

Both the VCL and FMX are both designed to operate in the main thread only, creating/updating UI elements in another thread is quite likely to cause you problems.

[havrlisan 3]

8 hours ago, Vincent Parrett said:

Both the VCL and FMX are both designed to operate in the main thread only, creating/updating UI elements in another thread is quite likely to cause you problems.

Sorry, I obviously should have pointed out that Thread 1 is the main thread, but it shouldn't matter since the problem is related to Rtti.

Update: I'm aware that one solution for this problem is to wait for one thread to finish using Rtti, then let the other continue. That is what I did, but I still want to hear your thoughts on this, and whether this is a bug that should be reported, or not.

Edited January 18 by havrlisan
update

[Dalija Prasnikar 1341]

2 hours ago, havrlisan said:

That is what I did, but I still want to hear your thoughts on this, and whether this is a bug that should be reported, or not.

Yes, this looks like a classic deadlocking bug. One thread locks one lock, while another locks other and they are dead in the water. It should be reported.

 

Possible workaround would be acquiring context at the application initialization (with KeepContext) and releasing it on shutdown. That way you would prevent EnsurePoolToken called from KeepContext to lock PoolLock. 

[havrlisan 3]

5 minutes ago, Dalija Prasnikar said:

Possible workaround would be acquiring context at the application initialization (with KeepContext) and releasing it on shutdown. That way you would prevent EnsurePoolToken called from KeepContext to lock PoolLock. 

I don't think acquiring the context would prevent the above call stacks from happening though:
The first stack goes through System.Classes:BeginGlobalLoading, which directly calls EnsurePoolToken and is not avoidable.
The second stack loads RTTI attributes with System.Rtti:ConstructAttributes method, which directly calls System.Rtti:Invoke method. In that method, TRttiContext is created right at the start.

[Dalija Prasnikar 1341]

10 minutes ago, havrlisan said:

The first stack goes through System.Classes:BeginGlobalLoading, which directly calls EnsurePoolToken and is not avoidable.

Yes, it will call EnsurePoolToken, but if the global token already exists it will not lock PoolLock while it tries to create one.

[havrlisan 3]

4 minutes ago, Dalija Prasnikar said:

Yes, it will call EnsurePoolToken, but if the global token already exists it will not lock PoolLock while it tries to create one.

You're right, I see it now. Are there any downsides to that approach? And if not, is there a reason Embarcadero did not do that in initialization and finalization sections?

Here's the link to the QP issue: https://quality.embarcadero.com/browse/RSP-44207

[Dalija Prasnikar 1341]

1 hour ago, havrlisan said:

You're right, I see it now. Are there any downsides to that approach? And if not, is there a reason Embarcadero did not do that in initialization and finalization sections?

Only problem you can face is if you are using dynamically loaded packages as the RTTI for that package will point to invalid memory. But I am not sure how relevant is that to the macOS as I don't know if you can even use runtime packages there.

1 hour ago, havrlisan said:

Here's the link to the QP issue: https://quality.embarcadero.com/browse/RSP-44207

Thanks! I added the test project.