Jump to content

Dalija Prasnikar

Members
  • Content Count

    1049
  • Joined

  • Last visited

  • Days Won

    91

Posts posted by Dalija Prasnikar


  1. 1 hour ago, Roger Cigol said:

    ..... and I am still hoping that with the new C++ approach that will eventually come there will after that eventually be Android 64 bit C++ support.   

    This work on modernization is the foundation as potential future support for Android and iOS requires more modern toolchain.

    • Like 1

  2. 4 hours ago, Rollo62 said:

    Yes, you're right. This statement is a little more clear, but still missing the clear explanation of what implications to expect for Delphi.

    There are no implications for Delphi. this is solely C++ Builder issue.

     

    AFAIK, C++ Builder side is going through a major upgrade https://docwiki.embarcadero.com/RADStudio/Athens/e/index.php/BCC64X and this work had a priority over supporting iOS and Android (which never had 64-bit compiler required for distribution through Google Play Store)

    • Like 2

  3. 2 hours ago, Rollo62 said:

    Does that mean, that the new Delphi 12.0 Athens officially is NOT able to fully support Android 13 and iOS 16 ( 32 and 64 Bit ) ?

    Delphi 12 supports those platforms. It is probably left over text from version 11, where You needed at least 11.3 to run applications on Android 13.

     

    I haven't tested it on iOS, but Delphi 12 definitely supports Android 13, including uploading on Google Play.

     

    • Like 1

  4. 3 hours ago, William23668 said:

    get results faster

    CPU is a finite resource. You will not get more of it, by throwing in more threads. If there are more threads than CPU cores, those threads will compete with each other for CPU time, and switching between threads also costs some CPU time. So after you overload the CPU, more threads you add the slower it will work.

     

    Only if threads are doing some I/O bound work, then you can have more such threads than CPU cores and possibly whatever you are doing can finish in less time. In other words if the thread spends most of the time waiting for some slow I/O operation, then such thread will not fully utilize CPU core and other threads can do useful work in the meantime on that core. However, even in I/O bound operations, there are limitations and if those threads are competing for the same resources, then again, the whole process will run slower.

     

    Imagine that CPU core is a shovel and threads are workers. In CPU bound work workers need to dig some ground. If you have same amount of shovels as workers than each worker will use the shovel all the time without having to give it up and it will be able to do the digging at full speed. If you have more workers than shovels, then workers will compete for that shovel. And it will not be in a way that some workers will use the shovel all the time and others will do nothing, but each worker will get very short time to use the shovel, maybe only taking one or two loads, and then it will have to give the shovel to the other worker that is waiting. But transferring shovel from one worker to another takes time, and at the end everything will run slower than if you have only one worker per shovel.

     

    On the other hand, if the worker also needs to use a pickaxe, then while worker is using the pickaxe, he does not need the shovel, and someone else can use it until the worker is done with the pickaxe. This is example of I/O bound work, where pickaxe is some I/O resource (network, disk...). Again, if you have more workers that compete for the pickaxes, the whole thing will work slower than single worker using that pickaxe.

     

    Now, this is simplified example. When it comes to your application running, it is not just your application that uses the CPU cores, but OS and other processes and applications are also using them, so you also need to take those into account. Now finding the right balance for the actual work can be hard and will depend on other parameters, and usually you don't have to optimize that much. But in simple terms, you definitely don't want to have more threads working at the same time than CPU cores, if those threads are doing CPU bound work. Commonly using as much threads as CPU cores or one less, will do - you can experiment with that, but again this is something worth pursuing for most applications. If you are writing something that will run on specific hardware then you can more easily optimize, but if you need to make it work across different ones, then good optimization for one may be a bad optimization for another.

    • Like 3
    • Thanks 1

  5. 1 hour ago, FabDev said:

    What is the better code to free a shared resource between thread (lStrings.free) :

    The first example is the correct one. The second one is not, as TThread.Queue will run after lStrings instance is being destroyed. Also it is extremely important that you pass nil to the TThread.Queue like you did because passing the current thread instance TThread.Queue(TThread.CurrentThread, would remove queued procedure when the thread has finished running and if that happens before queued code had the chance to run there would be a memory leak. This is less of a problem with tasks, as their threads are reused, although theoretically, such situation can still happen. But if you replace TTask with anonymous or custom thread then such situation is more likely to happen, especially if there are no more code in a thread after calling Queue that could delay its destruction. 

     

    However, the second example would also work if you would use TThread.Synchronize instead of Queue ad Synchronize is blocking call and Free will be executed only after code inside Synchronize is executed. 

    • Thanks 1

  6. 18 minutes ago, Yaron said:

    Most of the strings I use are WideString in order to support Unicode text in Delphi 7, not only that, all of my base visual components are based on the TNT Unicode library (e.g. TTNTForm, TTNTListBox, TTNTStringList, etc),

    Well, that could be a problem if those components are not supported on newer Delphi versions. I don't know because I am not familiar with them.

     

    If you do need to replace those components you can make substitute components that will allow loading of the project and will give you if not the same then similar functionality.

     

    List all the components you need and find appropriate functional replacement from the Delphi version you plan to upgrade. Then create package your own set of components that will have the same names as TNT components and inherit from appropriate components in new Delphi. Then add all the properties they have that might be different. You will also need to hook up and implement appropriate functionality behind those properties, unless they are functionally unnecessary and you just need to have dummy property to allow loading of a project in new Delphi. This set of components and controls will allow you to open the project in new Delphi version and use those as replacement for TNT components without the need to change form dfm files.

     

    It is important to make all those preparations on small test project so that you have everything working properly before you move your actual project.

     

    Unicode Delphi also has a WideString, so in theory you could keep using it, but if you are using WideString to interact with TNT components you may need to replace those to generic string type in new Delphi instead of introducing WideString properties to your TNT replacement components.

     

     

    • Like 1

  7. 14 hours ago, Yaron said:

    Is there a way for me to continue developing/fixing the current Delphi 7 version while slowly fixing the code (maintaining Delphi 7 compatibility)

    Like @Remy Lebeau said, start by using UTF8String for strings that hold UTF-8 data. It exists in Delphi 7, too. And doing that will make your transition to Unicode Delphi smoother. You will also be able to more easily verify that you haven't broken anything while doing changes in Delphi 7. This should be your starting point.

     

    Another thing you can do is make a short demo application where you will use the most common string handling patterns from your code and then you can transform and move that code to Unicode Delphi. If something goes wrong in that small demo you will be able to more easily understand what you need to do and how.


  8. 29 minutes ago, Yaron said:

    I thought of doing a blind search and replace from "String" to "AnsiString".

    Don't do that. This is the wrong approach. Just follow the hints from compiler. It will tell you where you need to do something about string conversion. For most of the code you will not need to change anything regarding strings. And if you do blind search and replace you will create more issues in your code than you will solve potential problems.

     

     


  9. 34 minutes ago, havrlisan said:

    LKey will always end up with whichever value is in the last loop. Does this mean that declaring a variable inside a loop will only reinitialize it every time at the same address?

    This is expected behavior. There is only single address behind inline variable declaration, not multiple ones.

     

    Also captured variables are not stored in the stack, but on a heap. Anonymous methods are basically defined as interfaces with a single method - Invoke - implemented by a hidden reference counted class, and captured variables are stored as fields of that class. When an anonymous method is accessed, an instance of that class is constructed behind the scenes, and it is kept alive through reference counting for as long as is required by the anonymous method it wraps. Because you have a loop you will only see the last value in that captured LKey variable because there is only one anonymous method instance created which you are adding to the list again and again with only one captured LKey field.

    • Like 2
    • Thanks 3

  10. 14 hours ago, luciano_f said:

    As I already mentioned, there is no reason not to have a 64-bit IDE since the whole world is moving towards this, let's get to the Facts


    Lazarus 64 Bit
    Visual Studio 64 Bit

    Android 64 Bit only
    MacOs 64 Bit only

    The whole world does it is not much of an argument. 

     

    Anyway, Android Studio is a Java application and it is rather memory hungry. Just starting it will gobble up 500MB of memory. Opening very small project will raise that to 1.5 GB. Comparing to Delphi which uses 116 when started and about 400MB when you open small project (similar to one in AS). So Delphi with opened project consumes less memory than AS without any. When you build that small project in AS it will go over 3 GB like it is nothing. Of course it only comes in 64-bit variant.

     

    macOS OS has been only 64-bit for over a decade. They have been pushing all applications to 64-bit from the very start and since 2019 macOS no longer supports running 32-bit applications. It is wonder they lasted that long as Apple is not much concerned with backward compatibility and the ability that their customers can run something or not. Or whether they will be forced to buy new hardware. Actually everything Apple does is to force people to buy new hardware. 

     

    Visual Studio got 64-bit version less than 2 years ago. If the MS with all their resources haven't done that before nor though it was absolutely necessary, then certainly there is no reason why Embarcadero would have to jump into 64-bit immediately.

     

    I am not going to comment Lazarus, as I don't know to much about it.

     

    Quote

    Who knows, maybe the day Microsoft says it will end support for Win32 will see Embarcadero in shame.

    I seriously doubt MS would do that. Maybe when they switch to 128bit Windows.

    • Like 3
    • Thanks 1

  11. 7 minutes ago, Attila Kovacs said:

    So basically, my approach to create/try-finally/free the TRtttiContext, which is advocated by some, is wrong because of the faulty library, and I should instantiate one somewhere at the beginning?

    Instantiating TRttiContext at the beginning would not help in this case as the problem is in additional lazy loading. Also not creating global TRttiContext is also affected. Whatever you do it will be broken.

     

    However, depending on the Delphi version you are using, creation of global TRttiContext solves some other threading issues. And it is also faster than doing create... try..finally.. free approach. 

     

    You can patch System.Rtti to fix the issue.

    • Like 1

  12. 15 hours ago, Attila Kovacs said:

    I came across this ineresting post https://stackoverflow.com/questions/27368556/trtticontext-multi-thread-issue after getting an error report over an AV in System.Rtti.MakeClosure

    and I'm wondering, according to the answers from @Stefan Glienke and @David Heffernan if the "data := nil;" in System.Rtti.LazyLoadAttributes,MakeClosure should not be the first after

    the second check?

     

    First of all, when asking about particular issue specifying Delphi version is mandatory. System.Rtti is being updated for literally every release (including updates). If you want us to comment on source code we need to make sure that we are looking at the same code.

     

    The code in question is perfectly fine. But the whole thing is broken in another place. Namely procedure TRttiInstanceType.ReadPropData; and other similar methods which use boolean for checking, but without double checked locking. This can causes calling LazyLoadAttributes twice which then can corupt the whole thing in that code. For instance I would get a crash on nil Finalizer instance.

     

    if FReadPropData then
        Exit;
    
      p := PByte(TypeData.PropData);
    
      TMonitor.Enter(Package.FLock);
      try
       // Here we would need to check FReadPropData again end exit if it is true 
        classic := ReadClassicProps;
        ext := ReadExtendedProps;
    
        FProps := SubtractClassic(classic, ext);
    
        FAttributeGetter := LazyLoadAttributes(p);
        FIndexedProps := ReadIndexedProps;
        FReadPropData := True;
      finally
        TMonitor.Exit(Package.FLock);
      end;
    end;

     

    • Like 1

  13. 52 minutes ago, Kas Ob. said:

    Anyway, i sounded my opinion about such GDI leak from AlphaSkin based on my point of view, and still see the cause by overflowing buffers or overwriting memory, to me the logical and easiest way to explain such behavior based on what OP said all happen with introducing ICS with OpenSSL and threading, and the fact if this overflowing was having anything else than zero's AlphaSkin code should generated hell of exceptions, threading problem can't be so consistent with zeroing memory, this leads me to deduct the only place that could make this happen with Delphi 7 and it outdated way to handle strings with different encoding coming from OpenSSL input and output, that is it.

    There is still a lot of code running in that thread that we haven't seen and don't know how it interacts with other code and might cause memory corruption for various reasons. But the string encoding/decoding code we have seen is not it, no matter how outdated or not.


  14. 37 minutes ago, Kas Ob. said:

    This is not a problem as long the input from Delphi or from a system like Windows OS which traditionally couldn't and didn't in the past support these UCS-4 aka UTF32, but now windows does support them like Android, and this makes a problem.

    Take an example this pizza slice emoji 🍕, in the past there wasn't away to input from keyboard but now windows does support it, this pizza slice is 3 bytes in Unicode, but it is 4 bytes in UTF8 and UTF16, and Delphi with its String or WideString will not be able to handle it, but again since long time ago

    Delphi can perfectly handle pizza slice emoji with either UTF8String or WideString. How do you think it was handling Chinese characters that were spanning two UTF-16 code units?

     

    Unicode encoding is a standard no matter what OS you are using. The most that can happen is that you cannot properly display the character because you are lacking appropriate font.

     

    Now I cannot claim that Delphi 7 encoding/decoding functions are capable of handling all those correctly as I haven't tried, but even in Delphi 7 those functions could be replaced with the ones that handle them properly.


  15. 16 hours ago, Kas Ob. said:
    17 hours ago, Dalija Prasnikar said:

    If Unicode character would require 4 bytes in UTF8 encoding, then it would also require two wide characters (UTF16 encoding) which means reserved number of bytes for such character would be 2 * 3 = 6.

    True, but, UTF16 is either one or 2 of 16bit units ( it is called unit in Wikipedia), so it is either 2 bytes or 4 bytes, why this wasn't ever a problem with all versions of Delphi ?,

    What problem?

     

    In context of wide string we can talk about wide characters where each such character stores one 16-bit code unit. This is important for determining the length of the wide string which is measured in number of wide characters - 16-bit code units.

    So full Unicode character can be stored either in single or two 16-bit wide characters, which means that for characters that span two wide characters reserved space will be 2 * 3 which is more than enough to store that whole Unicode character in UTF8 representation which can be maximum 4 bytes. There is no Unicode character that fully fits into single 16-bit code unit that requires more than 3 bytes when encoded in UTF8.

     

    16 hours ago, Kas Ob. said:

    The is simply because all the input and output where either internally generated data or by the OS ( mostly Windows), Windows API always refer to them as Wide, anyway this shouldn't a problem as long as the input and output is from Delphi application or the Windows, but in this very case the data are coming from remote place over the wire who knows the origin, also being handled by OpenSSL, so assuming it is will be 2 bytes UTF8 or even 2 bytes UTF16 is at least doubtful.

     

    I have no idea what you want to say here. 

     

    16 hours ago, Kas Ob. said:
    17 hours ago, Dalija Prasnikar said:

    Also if the supplied buffer is not enough then the called conversion function would fail (without causing buffer overrun) and would return 0 and such scenario is fully covered by UTF8Encode function.

    I really looked and looked then looked again, i just don't see failing point in both functions Utf8Encode and UnicodeToUtf8, this UnicodeToUtf8 though does trim but never fail, will return 0 if the input is an empty string other than that it will trim the result according to supplied buffer if it is shorter than needed, but will not fail.

    I was in a hurry and I took wrong turn while looking at code. Yes, you are right, the conversion function does not fail. The main point still stands and it does not cause buffer overrun. And again coming from UTF8Encode, conversion buffer allocates enough space for successful conversion of WideString content.

     

    So the bug in the code we are discussing here does not come from handling strings and their buffers, at least not in the code that was presented here.


  16. 1 hour ago, Kas Ob. said:

    How is this not wrong if this function assumes a UTF8 char can be only 3 bytes max instead of 4 ?

    What else is wrong with that conversion function, have you seen these two calls in one line of code, are we juggling strings left and right through many types from WideString, AnsiString then UTF8String with wrong max length and expect it to not be wrong, because it does work sometimes ?

    image.thumb.png.251a299ae94833f9be8edc2b5307b97a.png

    The function is not wrong. If Unicode character would require 4 bytes in UTF8 encoding, then it would also require two wide characters (UTF16 encoding) which means reserved number of bytes for such character would be 2 * 3 = 6. Also if the supplied buffer is not enough then the called conversion function would fail (without causing buffer overrun) and would return 0 and such scenario is fully covered by UTF8Encode function.


  17. 1 hour ago, TazKy said:

    A sinister thought could be that one day Google could force every app being considered for its play store be exclusively built using Android Studio only due to its every restricting security requirements with each version release.

    That would not happen since they support C++ in Android Studio. And such compiled code is no different than Delphi code.

    • Like 1

  18. 45 minutes ago, Kas Ob. said:

    Also was there a UTF8String in Delphi 7 ? and how it being handled in , what did Length() return, length in bytes or in chars ?

    Yes, there is a UTF8String in Delphi 7. 

     

    7 minutes ago, uso said:

    Off-topic: UTF-8 is 1 to 4 byte(s) per char depending on the respective code point.

    That is not what character means. It is how much code units are required for encoding Unicode code point in specific encoding.

     

    Unicode character is called a code point and depending on encoding each code point can occupy from 1 to 4 bytes https://stackoverflow.com/questions/2241348/what-are-unicode-utf-8-and-utf-16

     

    Character in Delphi can have different meanings:

     

    One is Char type which can have size of one or two bytes depending on Delphi version (pre 2009 or 2009+). There are also other character types which can have different byte sizes.

     

    Another one is used in terms of character type used in various string types: character size for AnsiString and UTF8String in all Delphi versions is one byte, for default string type in pre Unicode versions size of character is one byte, for Unicode versions size is two bytes.

     

    Length function always returns the size in characters for particular string type which means the actual number of bytes will depend on the character size for that string type. for UTF8String length will always equal the number of bytes because size of character in UTF8String is one byte, regardless whether some Unicode characters (code points) require more that one byte when encoded.

     

    19 minutes ago, uso said:

    It is not a trival task to get the exact byte size of an UTF-8 coded string.

    Yes it is. Just call the Length function on it.

     

    20 minutes ago, uso said:

    You need to put into consideration every code point. If you know your input languages, you may be able to approximate it - most of the time..

    You only need to consider how many bytes are in codepoint if you are parsing such data byte by byte, where accessing UTF8String by index will not necessarily give you complete information about Unicode code point stored as you will get only one byte of it.

    • Thanks 1

  19. 14 minutes ago, Angus Robertson said:

    ICS components could set the MultiThreaded property automatically, but the developer would still need to call IcsWndControl.ProcessMessages somewhere in the thread or the code would just stall, except in the simplest of applications.

    Yes, you are right.  I am trying to solve one problem, while forgetting the broader picture.  Although it might be easier to figure out what is wrong if the thread would just stall, comparing to having random issues somewhere else in the application.

    • Like 1
×