John Kouraklis 93 Posted March 16, 2020 Hi, I've got a class and it has a property TDictionary<..,..>. I want to use RTTI to access the key and value of the dictionary. I am able to get the property but I can't figure out how to access the key and values. I use this approach: var genericDictionary: TDictionary<TKey, TValue>; propValue: TValue; ... propvalue.ExtractRawDataNoCopy(@genericDictionary); ---> Here I can see in the debugger that genericDictionary has the items and the correct count Then, I get AV when I try to iterate the Keys because TKey is an abstract class. Anyone can help? Thanks Share this post Link to post
Remy Lebeau 1330 Posted March 17, 2020 (edited) I imagine trying to access the content of a TDictionary via RTTI will be very difficult, if not down-right dangerous due to the extra work needed to deal with managed types and such. You would be better off simply retrieving the TDictionary object pointer from the property (ie, using "propvalue.AsType<TDictionary<TKey, TValue>>" or "propvalue.AsObject as TDictionary<TKey, TValue>") and then access its content via normal TDictionary methods, not via RTTI at all. Edited March 17, 2020 by Remy Lebeau Share this post Link to post
John Kouraklis 93 Posted March 17, 2020 Hi Remy, I tried that (.AsType....) before as well; I run to the same problem of using TKey which is an abstract class although I can see in the debugger that the object points to the content of the dictionary.] Share this post Link to post
Remy Lebeau 1330 Posted March 17, 2020 15 minutes ago, John Kouraklis said: I tried that (.AsType....) before as well; I run to the same problem of using TKey which is an abstract class although I can see in the debugger that the object points to the content of the dictionary.] Then please show your actual code. It sounds like you are trying to instantiate TKey objects, which you should not be doing at all. You don't need to create a TKey object in order to enumerate a TDictionary. Use the TDictionary.Keys and TDictionary.Values properties, or the TDictionary.GetEnumerator() method. Share this post Link to post
John Kouraklis 93 Posted March 17, 2020 I found out that TKey in my code is declared (for unexplained reasons) in Data.DBXCommon with is irrelevant. Maybe that is the cause of the AVs I get. But now I don't seem to be able to declare TKey as data type although I include System.Generics.Collections. How do I declare it in a unit? It is originally declared using $HPPEMIT (Generics.Collections, line 750) Share this post Link to post
John Kouraklis 93 Posted March 18, 2020 Ok, now the more important problem as stated in the beginning. In this code, genDict throws an exception although propValue is correct program Project1; {$APPTYPE CONSOLE} {$R *.res} uses System.SysUtils, System.Generics.Collections, System.Rtti; type TObjDic = TObjectDictionary<string, string>; TMClass = class private fMC: TObjDic; public constructor Create; destructor Destroy; override; property MC: TObjDic read fMC write fMC; end; constructor TMClass.Create; begin inherited; fMC:=TObjDic.create; end; destructor TMClass.Destroy; begin fMC.Free; inherited; end; var mc: TMClass; ctx: TRttiContext; rType: TRttiType; rProp: TRttiProperty; propValue: TValue; genDict: TObjectDictionary<string, TValue>; begin mc:=TMClass.Create; rType := ctx.GetType(mc.ClassInfo); rProp:=rType.GetProperty('MC'); // In this code, this may seem redundant but I use this code is part of // bigger codebase propValue:=rProp.GetValue(mc); genDict:=nil; genDict:=propvalue.AsObject as TObjectDictionary<string, TValue>; if Assigned(genDict) then begin for var str in genDict.Keys do begin // For demo var f:=str; end; genDict.Free; end; mc.Free; end. Share this post Link to post
Remy Lebeau 1330 Posted March 18, 2020 (edited) 17 hours ago, John Kouraklis said: Ok, now the more important problem as stated in the beginning. In this code, genDict throws an exception although propValue is correct You have a mismatch in your code: Quote type TObjDic = TObjectDictionary<string, string>; ... var ... genDict: TObjectDictionary<string, TValue>; begin ... genDict:=propvalue.AsObject as TObjectDictionary<string, TValue>; ... Do you see it? String <> TValue. You need to fix your declaration of the genDict variable to match the TObjDic type. In fact, why are you not using TObjDict itself in the declaration of genDict? type TObjDic = TObjectDictionary<string, string>; ... var ... genDict: TObjDic; begin ... genDict := propvalue.AsObject as TObjDic; ... Edited March 18, 2020 by Remy Lebeau Share this post Link to post
John Kouraklis 93 Posted March 18, 2020 The whole code is part of a serialiser, so the dictionary type is unknown to the RTTI code. In the code above, I replaced TKey with string because I couldn't find the declaration of TKey. Share this post Link to post
Remy Lebeau 1330 Posted March 18, 2020 (edited) 4 hours ago, John Kouraklis said: The whole code is part of a serialiser, so the dictionary type is unknown to the RTTI code. Generics are not easy to serialize manually because of the nature of their dynamic types. You will likely have to resort to something like the following (untested, but should give you an idea of what is involved): var mc: TMClass; ctx: TRttiContext; rType: TRttiType; rProp: TRttiProperty; rMethod: TRttiMethod; rKeyField, rValueField: TRttiField; propValue, methValue, genKey, genValue: TValue; genDict, genDictEnum: TObject; genPair: Pointer; begin mc := TMClass.Create; try { if Assigned(mc.MC) then begin for var pair in mc.MC do begin // use pair.Key and pair.Value as needed ... end; end; which is actually this behind the scenes ... if Assigned(mc.MC) then begin var enum = mc.MC.GetEnumerator; while enum.MoveNext do begin var pair = enum.Current; // use pair.Key and pair.Value as needed ... end; end; } rType := ctx.GetType(mc.ClassInfo); // rType = TMClass rProp := rType.GetProperty('MC'); propValue := rProp.GetValue(mc); genDict := propValue.AsObject; if Assigned(genDict) then begin rType := rProp.PropertyType; // rType = TObjDict rMethod := rType.GetMethod('GetEnumerator'); methValue := rMethod.Invoke(genDict, []); genDictEnum := methValue.AsObject; rType := rMethod.ReturnType; // rType = TDictionary<TKey, TValue>.TPairEnumerator rMethod := rType.GetMethod('MoveNext'); rProp := rType.GetProperty('Current'); rType := rProp.PropertyType; // rType = TPair<TKey, TValue> rKeyField := rType.GetField('Key'); rValueField := rType.GetField('Value'); methValue := rMethod.Invoke(genDictEnum, []); while methValue.AsBoolean do begin propValue := rProp.GetValue(genDictEnum); genPair := propValue.GetReferenceToRawData; genKey := rKeyField.GetValue(genPair); // genKey.TypeInfo = TypeInfo(string) genValue := rValueField.GetValue(genPair); // genValue.TypeInfo = TypeInfo(string) // use genKey and genValue as needed ... methValue := rMethod.Invoke(genDictEnum, []); end; end; finally mc.Free; end; end. Quote In the code above, I replaced TKey with string because I couldn't find the declaration of TKey. Because it doesn't exist. For a dictionary, TKey and TValue are Generic parameters of the TDictionary class. They are not standalone concrete types, like you are thinking of. Edited March 18, 2020 by Remy Lebeau 1 Share this post Link to post
John Kouraklis 93 Posted March 18, 2020 @Remy Lebeau Wow....Amazing and complicated stuff. So I really need to drill down to the enumerator to get hold of the pairs. Thanks a lot for this!!! You mentioned that it might be unreliable to do this with generics. Do you think the app may crash for more complex dictionaries? 22 minutes ago, Remy Lebeau said: Quote In the code above, I replaced TKey with string because I couldn't find the declaration of TKey. Because it doesn't exist. For a dictionary, TKey and TValue are Generic parameters of the TDictionary class. They are not standalone concrete types, like you are thinking of. And out of curiosity, how does the compiler recognise TKey then? It must be from the HPPEMIT directive, right? Share this post Link to post
Remy Lebeau 1330 Posted March 19, 2020 (edited) 2 hours ago, John Kouraklis said: @Remy Lebeau Wow....Amazing and complicated stuff. So I really need to drill down to the enumerator to get hold of the pairs. Yes, if you can't specify the actual TKey/TValue Generic parameters up front at compile-time when defining how you access the dictionary object from the Sytem.Rtti.TValue record. In which case, you can't use a standard "for..in" loop to enumerate the dictionary. All the necessary type information is lost once you access the dictionary object via the System.Rtti.TValue record if you can't type-cast the object. So yes, this is the only way I can think to enumerate the dictionary object using only RTTI and not caring what the actual Generic parameters are. I mean, if you really needed the RTTI for those types, you could take the dictionary object's ClassName() string, parse the typenames between the angle brackets, and resolve them using TRttiContext.FindType(), but that won't help you to enumerate the dictionary at all since you still need access to its enumerators (either via its GetEnumerator() method or its Keys+Values properties). Quote You mentioned that it might be unreliable to do this with generics. Do you think the app may crash for more complex dictionaries? I really couldn't say. Probably not, if you use the RTTI correctly and handle the enumeration correctly. Hopefully the RTTI and Sytem.Rtti.TValue record will handle the complexities for you. Quote And out of curiosity, how does the compiler recognise TKey then? It doesn't, outside of the implementation code for the TDictionary class, where the TKey and TValue Generic parameters have meaning. Outside of the class, they don't exist. They are part of the class type itself, for instance TDictionary<string, int> and TDictionary<int, char> are separate and distinct types with no relation to each other. Quote It must be from the HPPEMIT directive, right? No. {$HPPEMIT} has no effect whatsoever on how the Delphi compiler processes code. {$HPPEMIT} merely outputs an arbitrary line of text to a C++ .hpp header file, if one is being generated while the Delphi compiler is parsing the code. Curious - what version of Delphi are you using? I have RTL sources up to XE3, and there are no {$HPPEMIT} statements at all in System.Generics.Collections.pas in those versions, so it must be something added in more recent versions. Edited March 19, 2020 by Remy Lebeau 1 Share this post Link to post
John Kouraklis 93 Posted March 19, 2020 Remy, thanks for the explanation and for sharing your knowledge 👍. I admit that even if I had gone all the way down to retrieve the enumerators I would never have thought to Invoke them. I use 10.3.3 and the first instance of TKey in System.Generics.Collections is this: ------> This is line 748 (*$HPPEMIT END OPENNAMESPACE*) (*$HPPEMIT END ' template<typename TKey, typename TValue>'*) (*$HPPEMIT END ' inline __fastcall TPair__2<TKey, TValue>::TPair__2(const TKey AKey, const TValue AValue) :'*) (*$HPPEMIT END ' Key(AKey), Value(AValue)'*) (*$HPPEMIT END ' {}'*) (*$HPPEMIT END CLOSENAMESPACE*) [HPPGEN(HPPGenAttribute.mkNonPackage)] TPair<TKey,TValue> = record Key: TKey; Value: TValue; constructor Create(const AKey: TKey; const AValue: TValue); end; // Hash table using linear probing TDictionary<TKey,TValue> = class(TEnumerable<TPair<TKey,TValue>>) private type TItem = record HashCode: Integer; Key: TKey; Value: TValue; end; TItemArray = array of TItem; Share this post Link to post
Remy Lebeau 1330 Posted March 19, 2020 (edited) In that TPair record, TKey and TValue are Generic parameters, they belong to the TPair type itself. In C++, that TPair record type gets renamed to TPair__2 because of the Generic parameters. When the Delphi compiler generates a C++ .hpp file, it will declare a constructor inside that type. But since the type is renamed for C++, the HPPEMITs are being used to provide an implementation for that constructor using the new name. I have never seen the [HPPGEN] attribute before, though. It is not documented. Edited March 19, 2020 by Remy Lebeau 1 Share this post Link to post
John Kouraklis 93 Posted March 21, 2020 Thanks Remy--Always good to learn Share this post Link to post
Andrea Magni 75 Posted November 2, 2021 Hi @John Kouraklis and @Remy Lebeau, thanks for this topic that helped me too. Just wanted to add I think we should free the enumerator object (returned by TDictionary<>.GetEnumerator). I've used this approach in MARS REST library, as you can see here: https://github.com/andrea-magni/MARS/blob/41fd78e5416e38fad0f6a0451402a10824aa62c4/Source/MARS.YAML.ReadersAndWriters.pas#L222 Thanks again you both. Andrea 1 Share this post Link to post
Remy Lebeau 1330 Posted November 2, 2021 (edited) 9 hours ago, Andrea Magni said: Just wanted to add I think we should free the enumerator object (returned by TDictionary<>.GetEnumerator). Asked and answered on StackOverflow: Do I need to free the enumerator returned by GetEnumerator? Edited November 2, 2021 by Remy Lebeau Share this post Link to post
darnocian 72 Posted November 2, 2021 I wish RTTI embedded the generic type parameters and the TRttiType included a way to extract them. I've often had to rely on specific use cases and knowing that I can extract the types easily from known method return types as highlighted by the enumerator / pair example above... I've had times where I needed some really generic stuff and had to split the TWhatever<T> as a string and resolve the generic parameter however it is defined recursively - which is a bit annoying. Share this post Link to post
Andrea Magni 75 Posted November 3, 2021 14 hours ago, Remy Lebeau said: Asked and answered on StackOverflow: Do I need to free the enumerator returned by GetEnumerator? I saw that too. I meant in the code above a genDictEnum.Free is missing somewhere before mc.Free call. Thanks Share this post Link to post