Kas Ob.

@DelphiUdIT On side note: (after seeing you mention "mov ax, source" and "mov dest, ax") If you are going to open a ticket then report the hidden performance killer, it strikes randomly, due the simplicity of the compiler and its usage of specific 16bit operation instruction. from "3.4.2.3 Length-Changing Prefixes (LCP) " https://www.intel.com/content/www/us/en/content-details/671488/intel-64-and-ia-32-architectures-optimization-reference-manual-volume-1.html Most Delphi string operation does utilize 16bit loading the storing, the above LCP effect does occur when a these instruction aligned (or positioned at specific address 13h or 14h, repeating and reproducing it hard as it might be stay hidden until some changes in complete part of the project triggered a small shift, also very important thing, these instruction are generated by the Delphi compiler without any consideration for such alignment, so what happen in real life and when you benchmark that it keeps shifting up and down and differentiate from version to another from build to build, what we witness is close to steady result, but in fact the bottlenecks where shifted form place to another, fixing this will ensure better and more consistent performance in string handling in any Delphi binary produced. I believe this was the case not a long ago when i suggested to replace 16bit field fill with 32bit and let it overflow, the suggestion was outlandish unless that instruction hit specific address and then cause 6 or 11 cycle of stall ( full pipe stop), and it looks it was the case as Vencent observed.

If it could be done better it would had. ( not sure if that is correct English phrase) Anyway, you need to look to the hardware implementation as its basic building blocks, so you can't invent an instruction or algorithm easily, you either need huge hardware circuits which will cost more delay and other problem, or utilize already exist uops ! so here a more detailed look at MOVSQ https://uops.info/html-lat/AMT/MOVSQ-Measurements.html It does show that 17 or 18 micro ops used for one MOVSQ, and this is huge latency.

The compiler is generating very bad instruction, even calling RTL Move would be faster then using MOVSQ !! https://uops.info/html-instr/MOVSQ.html 4 cycles in theory but in reality it is 7-8 cycles on my CPU ! Suggestion find these and manually move/copy the fields, specially in tight loops.

I think the best way is to have a singleton, that initialized very early in the project (preferably right after MM and before forms), this singleton will hook TComponent.Create and Destroy, grabbing its address (value/pointer) and add it to internally managed list, this list have the component associated with an extra data, extra data could be a string, integer, record.. and even classes. This will solve/remove the need to manually managing this extra data (eg. string), also ensure memory integrity for this extra data without interfering with the associated component in anyway, on top of that helper(s) for specific components can be used for easier and simplified use.

From the latest RFC https://www.rfc-editor.org/rfc/rfc8259#section-2 Which wasn't the case in the older RFC(s), and even now it is still not defined concretely, protocols that digitally sign payloads require no spaces like ACME, as it should parse it before and still generate the same exact after parsing to match the hash.

In addition to Remy points, i want to focus on : that your JSON is not standard, and many parsers might see it as broken, JSON should be in compact form and doesn't/shouldn't have spaces.

If only it was this simple. and NO, that will not work, you are miss understanding how encryption work, and don't be offended, this is not for everyone. First you have to read a little how encryption modes work, https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation Second, you must understand and never forget one essential element in encryption, the internal state (the context) does advance with every encryption process performed, meaning .... well this will get a little complicated here... 1) If the Data length (in whole) less than one block, this the output is similar to CTR with IV=0, CTR doesn't have IV (to be exact) but have counter|nonce works as IV, but as it is 0 for the first block then it does work for this case with DevArt broken implementation. 2) If the Data length multiple is longer than 8 bytes, then for all full blocks (n*8) then it is pure CBC, and it is correct. 3) here comes the rest if there is more fewer bytes than 8 meaning [1..7] bytes when the length of Data (the original length ) was more than 8, in this case there is nothing out of the box can help you or able to decrypt it, Why? Because the internal state of the this last block is already unknown and depends on the last one, and this can't be fixed by using key or IV !!, from the wikipedia link you see these boxes indicating encryption, but the state is changed and can't be injected or altered, this is way more complicated then follow the wrong doing. As i said, you need an implementation and it need to be corrupted to be compatible, just to make sure you do understand what i am saying, it is CTR for less than 8 bytes CBC for all the 8*n, if there is remaining bytes then it is more like CFB mode, and in all cases the state is one !, meaning you can't switch or reinitialize cryptor/decryptor, you need the state to be chained as usual. You also didn't answer my question, about is it essential to be decrypted using different language/package/library, as you can use DevArt Encryptor to decrypt anything encrypted with it, it is wrong but consistent. Also look at this clean and simple implementation in C# (no extensive OOP or forced padding), https://gist.github.com/vbe0201/af16e522562b2122953206d8bbd1eb50#file-alefcrypto-cs-L368 at that line i marked, comes the adjustment to handle full blocks or xor after encryption for less than full block and this in theory should solve your problem, in case you want to go with this way, then build project and test cases for multiple values with multiple length and come back if you can't do it, and i will try to help. An you know what CTR that you found returning same result for less than 8 bytes is identical to CFB with IV=0, so it is CBC for all 8 bytes then the rest is handled as CFB and to be accurate CFB8 (as 8 bits) happen on bit level in case you faced this discriminations searching the web.

@dormky I do understand the problem now in all needed depth, i do understand the wrong implementation in TCREncryptor, and it is when DataHeader is set to ehNone, i can solve this and also you can. The buggy implementation is as follow : 1) CBC mode is implemented right, but fall short when the length of the data is not multiple length of blocks of 8 bytes. 2) for all the blocks with 8 it is right CBC, aka xor then encrypt. 3) when there is extra it is doing something wrong, it does encrypt then xor, and that why you are getting identical values with CTR mode when the data is shorter than 1 block, as CTR is not initialized and hold 0. Now, to your need: Up you mentioned Python and in your last post you are using C#, if this is not problem then why not use DevArt broken implementation to decrypt, notice that no matter what are you going to use, you need broken and altered implementation. So you have the ball, decide what and tell me and i will try to help.

Tomorrow will try to figure something for you, no promise though, after almost two hours digging into this, i can say the encryption is utterly broken and non standard CBC, it is wrong !

I don't have any experience with LockBox3, and the fact there is memory leak(s) is enough reason to stay away form it, Anyway.. 41 is Ord('A'), in other word it looks it didn't encrypt anything, to confirm try 'B', if you got 42 then either the library is broken or you are using it wrong !

Just to fix your problem and workarounds The hash of the password is generated from the WideString of encoded password so const PASSWORD = 'pass'; var keyBytes, PassBytes: TBytes; HashSHA1: THashAlgorithm; HashMD5: THashAlgorithm; begin PassBytes := BytesOf(@WideString(PASSWORD)[1], Length(PASSWORD) * SizeOf(WideChar)); HashSHA1 := THash_SHA1.Create; try HashSHA1.Initialize; HashSHA1.ComputeHash(PassBytes); HashMD5 := THash_MD5.Create; try HashMD5.Initialize; HashMD5.ComputeHash(PassBytes); keyBytes := HashSHA1.Hash + HashMD5.Hash; SetLength(keyBytes, 32); // doing it here is better, triming here 32 for BlowFish or we can later use specify length at 32 with SetKey finally HashMD5.Free; end; finally HashSHA1.Free; end; .... table2.Encryption.Encryptor.SetKey(keyBytes, 0, Length(keyBytes)); // length should be 32 but we don't want to overflow 1) Any hash library will do the same you are free to use your own 2) The length is critical so make sure you are feeding 32 bytes, (and no more ! as strangely enough it does affect the output, meaning the SetKey is not protected from overflowing) 3) There is two version of SetKey, one is bugged and wrong the other does work fine, use the one with the offset and require TBytes as parameter, the one with 3 parameters.

Also SetKey is buggy !!! and not only ! procedure TCREncryptor.SetKey(const Key; Count: Integer); begin SetKey(TBytes(@Key), 0, Count); // TBytes(@Key) instead of TBytes(Key) end; procedure TCREncryptor.SetKey(const Key: TBytes; Offset, Count: Integer);

Have a good weekend too ! and now i do understand your points, also it took me long time as i don't want to mess with packages installation, as MyDAC and UniDAC can't be installed together unless the DAC version is identical, anyway... 1) The i described building the key is working fine, i checked and it is fine. here a simple way to build the key and use it using only UniDAC classes var keyBytes: TBytes; HashSHA1 : THashAlgorithm; HashMD5 : THashAlgorithm; begin HashSHA1 := THash_SHA1.Create; try HashSHA1.Initialize; HashSHA1.ComputeHash(BytesOf('pass')); HashMD5 := THash_MD5.Create; try HashMD5.Initialize; HashMD5.ComputeHash(BytesOf('pass')); keyBytes := HashSHA1.Hash + HashMD5.Hash; finally HashMD5.Free; end; finally HashSHA1.Free; end; 2) The encoding and decoding using CBC without padding is fine as i described it, here your 'A' with ehTag and ehTagAndHash in order C544E5292C9C42A5B94FE279127029010796125FFF7BB9A7408E02812A51D5F81FC2053401075D2A C544E5292C9C42A5B94FE27912702901DF4BEED6024BC5FE214A4205B1EE9981DAF4C7C56DCD4CE23D88E2EE9568BA546C007C63D9131C1B while with none it is two bytes E2 Now what is going on, that is strange thing to encrypt without IV, in fact you can't and shouldn't not do that , that is wrong on so many levels !!!!! The IV is zeroed in this case, and this is what you are missing, simple like that. Anyway E2 is HEX you need to de-hex it first then perform decryption in CBC mode, may be i missed mentioning this, but hey.. i don't know if you don't know that encryption with DB should be either binary in blob or HEX in plain text fields. -------------- Additional information changing the code a little to insert two 'A' rows, to show the information leak with ehNone we have two rows have the same encrypted value two chars (value = 'E2'), leaking critical information, exposing the data is identical with ehTag we have C544E5292C9C42A5B94FE27912702901CA9469BBB8503499CDFA76E7E84DBE9BCB48BF853F5A2643 C544E5292C9C42A5B94FE279127029010F89F0C11CB311D592EA28B171C712B5C7725B0130002CE8 and with ehTagAndHash C544E5292C9C42A5B94FE2791270290152F2428A526B0345481FAB80E42E687379477B776DCD4CE23D88E2EE9568BA546C007C63D9131C1B C544E5292C9C42A5B94FE2791270290173F9E56EB70F416E1C5B5CAD820E2220584574896DCD4CE23D88E2EE9568BA546C007C63D9131C1B Now do you see the information leak ?!, yes the hash is the same with ehTagAndHash exposing that the encrypted data was the same !! So until DevArt fix this, everyone should only use "ehTag"

I am lost here, what metadata ? What "A" and where did you get it, are trying to encrypt ? Tried what ? there is no hardcoded IV. IV is randomly generated as i said and included in the bytes/blob/strings/widestrings/memo... encrypted data

Magic number is fixed 4 bytes value implemented in TCREncryptor as header its value "BAADF00D" in HEX, Sorry, for my writing way, well i am trying my best to explain, the problem i can't know where your knowledge limit about encryption to expand, so i write in a way i see serve the most. You have problem understanding symmetric encryption and its modes, you need to read more, and i will expand here what can help you. Yes all symmetric algorithms including BlowFish need specific block size, but it doesn't affect the output, the output is affected by the mode of the encryption, ...... i am not going to torture you with more text, Pad the god damn thing with anything to make it work ! is that better, but don't use any specific padding on decryption because it will fail ! , if your python library enforced to use padding then look for different library that allow no padding, and by padding i mean the algorithm defined in RFCs, not the block padding to ensure the block length, the encrypted data should be already multiple of 8. again the structure of the encrypted data, these concatenated KEY_GUID + IV +CIPHER [+HASH] KEY_GUID is fixed constant of 16 bytes declared internally declared and shipped unencrypted, it has no weight or value on encryption, starting with $C5 .. ending with $01, simply an identifier that the encryption done by DevArt library. IV well .. 8 bytes HASH depends on the Hash algorithm SHA1 or MD5 (property in the encryptor compenent) and if "ehTag" then it will not be included while "ehTagAndHash" means it is there CIPHER is INTERNAL_HEADER + PLAIN_DATA INTERNAL_HEADER is 8 bytes long, contains magic number, aka constant of 4 bytes with "$BAADF00D" in HEX , followed by 4 bytes contain the PLAIN_TEXT length To generate the key, calculate SHA1 and MD5 of the password then concat them, then use the first bytes to fit the key length you need ! simple as that IV you have it in above after KEY_GUID Don't use BlowFish implementation that require padding ! Hope that helps.

Padding has nothing to do with symmetric encryption itself, padding is standardized way to prevent data lose, many libraries force you to use padding out of best practice and to comply with Modern Security standards, in other words to prevent you from shooting your foot, but there is also libraries (if not all of them) that allow you to manually handle this. And you said you are using , in Delphi ? in Python ? DevArt library ? TCREncryptor doesn't use padding PKCS7 or any other, it does ship the length (4 bytes length) after magic number before the encrypted data and after the random IV. Now i do understand that, and i am trying to help by explaining, your lack of understanding padding and when to use it, will stop you, TCREncryptor doesn't use padding as you assume. No joy there, because no padding, if you insist then pad it with 0 or random, with CBC it will not make a difference, as you will already have the length as i explained, right before the data. Now to your problem, Are you trying to decode the data in full ? As i explained the encrypted data are structured and you only need to decrypt part of it, if you see the lines i pasted above The encrypted data start at KEY_GUID_LEN + IV_LEN , 16+8 bytes, at 24, the length is ... till the end in case of "ehTag" or leaving the length of the hash to the according algorithm when "ehTagAndHash", in my UniDAC version there is only two hash algorithms MD5 and SHA1. The encrypted data include INTERNAL_HEADER_LEN bytes which is 8 bytes, this is the magic number with the length (removing the need for padding), so after decryption you need to check for the magic number and get the length then trim the decrypted data accoring to the length.

So after looking more at the encryption code, i stand with claim above, but .. It could be my old and outdated sources, they either : 1) Should hashed the data along with the randomly generated IV, and leave it out of the encryption, or 2) They should have encrypted the hash along with the data, as this looks like a single mistake (old mistake), it might be changed or fixed, though this will break compatibility between versions, the code to fix this FEncryptor.EncodeBuffer(TValueArr(Result), KEY_GUID_LEN + IV_LEN, CipherLen - HashLen, TValueArr(Result), KEY_GUID_LEN + IV_LEN); // should be FEncryptor.EncodeBuffer(TValueArr(Result), KEY_GUID_LEN + IV_LEN, CipherLen , TValueArr(Result), KEY_GUID_LEN + IV_LEN); Any fix will break compatibility, so in my opinion they should deprecate "ehTagAndHash" and use it only for decryption, and introduce ehTagAndHash2 with either of two suggestions above, but this will break compatibility backward too. Of course any changes of the above suggestion need adjustment in the decryption too.

Yes, (though outdated). Also yes. Your problem is not with the key itself, the key literally as i mentioned above, concatenating SHA1 and MD5 of the password, also the mode is CBC, so less work there. Your problem is in the code in TCREncryptor.Decrypt (and TCREncryptor.EncryptBytes) , these are very nicely done and very secure with authenticated encryption (with TagAndHash enabled), along random IV for each and every encryption operation followed by magic number "$BAADF00D" indicates the length of the data, hence removing the need for padding, it is well done until they missed to include hashing the IV with data (!!) this will leak information about the encrypted data. You are more secure with only "ehTag" in DataHeader property than "ehTagAndHash", no leaked information about the data, preventing any differential analysis, but with this you will lose the authentication. I am sorry that i can't share any source code from a proprietary code licensed to me, as you seem you don't have it, but with the above few lines you can ask them to fix their mistake and answer your support ticket. Hope that helps.

However, also i looked at my UniDAC (also expired but way newer than MyDAC) and there is TUniEncryptor and as the DAC components are shared between all DevArt libraries, i think the algorithm is still that old and with not best practice or standard password-key derivation, as changing it will break compatibility unless it is explicitly had an option to switch to newer one, and i don't see such option, in the online help for the latest ones.

My MyDAC expired long time ago, and i never used their Password2Key algorithm, i always used a key directly with the encryption, but when needed to be password derived i used HMACSHA256, the algorithm to derive the key (in the my TCREncryptor) is outdated and not secure enough to provide 256bit security and may be not even enough for 128bit, the code does concatenate the SHA1 with MD5 of the password. Anyways, you would better contact their support for this question in case you can't recognize the used algorithm, while if you have the source as i do, then it should be in "procedure TCREncryptor.Password2Key;", but again in MyDAC there is no TMyEncryptor, there is only TCREncryptor, which means they did refactor this and may be the algorithm too.

Try this procedure TForm10.Button1Click(Sender: TObject); var MyReg: TRegistry; begin MyReg := TRegistry.Create; try MyReg.RootKey := HKEY_CURRENT_USER; if MyReg.KeyExists('\Control Panel\Cursors') then begin MyReg.Access := KEY_WRITE or KEY_READ or KEY_WOW64_64KEY; // KEY_WRITE or KEY_READ together or we can separate them per operation if MyReg.OpenKey('\Control Panel\Cursors', False) then begin //st := MyReg.ReadString(''); MyReg.WriteString('', 'Windows Black'); end; end; finally MyReg.Free; end; end;

I want to know and solve this, away from Windows Server 2003 and Windows XP rare existence, it is nice to have unified version to support them all, this single API stopped a punch of great and old software and split many repository. I don't see this as stupid or waste of time, if a time spent lead to a result in testing something, success or failure, for me this is well spent time.

Great and perfect and Thank you !

Right, i should have mentioned that may be as i read that second source you linked. According to this i have the it should (might) work for all starting from Windows XP SP2/SP3 and Server 2003 SP1, and that include 32Bit and 64Bit , as shown per declaration with 64bit size in that link, It is the same declared in DDK headers https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-kuser_shared_data also here explain the size of KSYSTEM_TIME https://osm.hpi.de/wrk/2007/08/getting-os-information-the-kuser_shared_data-structure/ And there a confession to make there is a condition i omitted in the code because i don't see it as big relevant, in 64bit there is a loop to establish that High2Time and High3Time are identical, this happen if updating one was happening at this very exact moment, i tried to slow my CPU by overloading up to 100%, and couldn't make this trigger, then tried to downclock and overclock CPU, as mine is Intel unlocked (model with "k"), yet even over more than an hour and in both cases could capture such case not even once, and i am not saying it is right to capture it, but i am saying statistically i satisfied, also even when they are not identical, meaning the result will be as much as it was like 100ns before, so no harm there, this can affect and important for kernel drivers, but can not be critical for any software in user mode.

Well, not exactly, QueryPerformanceCounter does have higher resolution but with lower accuracy, it is tied to the CPU cycles, at that low level, yes it has very high resolution, but when the CPU throttle or switch power profile it will start to return far from accurate result, also will be far from accurate in hibernate, GetTickCount on other hand depends or gets its clock timing using BIOS clock interrupt which in turn gets its time from the MotherBoard chipset, this one doesn't suffer from power profile effect, these chipset runs at constant frequency and they are very accurate but with lower precision than the CPU, due to their difference in base frequencies (CPU and motherboard chipset).