1. Debuggers, debuggers, debuggers - multithread handling is abysmal today
2. Make HighDPI actually works as intended - it is useless in a team as is - unless everyone runs the same scaling
3. Generics constraints for enumerated types to enable the use of conversion to/from ordinal values, use set operators, etc.
4. Native ARM64 compiler for Delphi for Windows (and Linux/Raspberry PI, but Windows has prio)
5. A 64-bit IDE that ensured that EMBT was dogfooding 64-bit VCL and RTL and raise the quality
As for AI, I wouldn't mind an AI that could look at code and suggest improvements - perhaps as a part of the static code analysis, or one that could explain "what does this code do".
I don't really need or want an AI to generate code. If it is a standard, a lib should already exist. If it doesn't and there is no standard that covers the need, I'd be happy to have an AI outline alternative approaches - but given that the wide scope of parameters that goes into a design, I think it is unrealistic to expect that it would come up with the ideal suggestion without us writing a huge requirement. The output of VLLM generators is extremely dependant on precise and accurate specification statements, and writing those are almost as hard as writing good code.
As for AI and privacy - keep your secret credentials separated from your code. Other than that, I don't think that many of us write code that truly needs to be hidden for secrecy reasons, although it is obvious that it will be necessary to ensure that privacy permeates the use of AI.