Delphi compiler need to be opensourced

[Dalija Prasnikar 1396]

9 hours ago, AlekXL said:

 

Ok, I've just checked. And must say, this is officially a lie (were you a rookie, I would call it mistake, but you're prominent, trusted member of the community, so take it)

 

Swig fork of Delphi is able to import C++ class -> Delphi class wrapped, through dll import.

Swig generates c++ proxy *.cxx from a header, and even able to specialize C++ template class (you need to write something like TEMPLATE_WRAP0(vector ,vecint,  int );

Swig flattens said class members to in the *.cxx and export this through __declspec(dllexport) (or statically, but delphi-swig cannot handle static linking)

Then, Swig generates pas stub , which imports flattened methods(stupidly, it does not add underscores to names, but this is easy fixed by

__DLLNAME name '  ==> __DLLNAME name '_ "replace all"

Additionally (default, but optional), Swig adds Delphi class wrapper, for the C++ class/specialized template class.

I've just regenerated, and compiled template example, from example.h example.i , with C++Builder and Delphi.. This example is simplistic, so I don't know how it would work it with complex c++ classes, but considering SWIG is serious tool, we can hope.

And here your splendid personality jumps out again. Yes, I made a mistake. People do that, you know, regardless of experience. After being knowledgeable in your field of work, ability to recognize and admit your own mistakes is one of important qualities for successful developers. If one is not able to own his own mistakes it is hard to expect that such person will allow others to make mistakes without making too much fuss about it. Your response shows that either you don't have that quality or you have really short fuse. And neither makes you a wonderful team mate.

 

And If you read my response you will see that I didn't included SWIG library (two and some years old) - I simply overlooked it. Last time I had to use C++ with Delphi was 8 years ago and it was no fun. I am glad that landscape has changed in that regard.

 

Delphi ecosystem is not huge (no matter how much we would want it to be), and ability to interact with libraries written in other languages can make a whole world of difference. You don't want to reinvent hot water if it is already out there. Also, you might use other toolsets besides Delphi and utilizing same libraries across those toolsets also counts a great deal.

[Dalija Prasnikar 1396]

7 hours ago, Joseph MItzen said:

And others could tell you stories of the issues that have been closed as "won't fix" or "by design". I remember Dalija Prasnikar got Marco to re-open several that he'd closed in the past (as far as I know they never actually got fixed though).

Baby steps 

 

If bug report is open, it does have greater chances for being fixed than if it is closed. Also not all bugs are easily fixable.

 

But one of the greatest issues, that for long period seemed like it will never happen, was introducing 8bit strings to all platforms. If nothing else, this makes me hopeful.

[AlekXL 8]

3 hours ago, Dalija Prasnikar said:

But one of the greatest issues, that for long period seemed like it will never happen, was introducing 8bit strings to all platforms. If nothing else, this makes me hopeful

old news. Best news is inline variables and deprecating ARC.

3 hours ago, Dalija Prasnikar said:

Your response shows that either you don't have that quality or you have really short fuse

Neither. "For they have sown the wind, and they shall reap the whirlwind", Hosea 8:7. You were quite critical about my demeanour, well, you always reap more than you've sown. Nature's law, God's Word, uncontestable anyway.

[Dalija Prasnikar 1396]

27 minutes ago, AlekXL said:

old news. Best news is inline variables and deprecating ARC.

It does not matter how old it is, it matters that things can change . 

 

Just like deprecating ARC at some point seemed highly unlikely. And for the record that is the change I didn't wished for.

[Dalija Prasnikar 1396]

30 minutes ago, AlekXL said:

Neither. "For they have sown the wind, and they shall reap the whirlwind", Hosea 8:7. You were quite critical about my demeanour, well, you always reap more than you've sown. Nature's law, God's Word, uncontestable anyway.

I am critical, because you are seeking collaboration and your attitude is making more harm than good. Not just for your particular request, but it is undermining other efforts in similar areas. I am not stranger to saying harsh words, but there is a difference between backing your words with actual arguments and empty phrases like "we love Delphi more than you do".

 

Feel free to criticize me when I do something wrong. I have no problem with that. 

 

I am an atheist and God has no meaning to me. For my actions I will be and I am accountable in this life only.

[AlekXL 8]

Marco was right, when he said that forced arc is no-go, because of legacy, and fragmentation.

But why would he say, that on-demand arc is not viable?

Quote

but an ARC-enabled object won’t easily coexist with a non-ARC-enabled one

they would coexist just fine.

Arc -> non arc

unsafeInstance := unsafe(safeInstance);// you are on your own

is valid, although not safe. Just add hidden 4-byte field to TObject, at negative offset, FRefcount. Like in string constants, make sure this  field is negative for non-ARC instances. According to the type of variable (class OR auto class), which currently manipulate instance, compiler would know, whether to call gen _Addref and _Release on assignments.

Quote

and we would need containers for both flavors of objects and would make it impossible to mix them

Generally, yes, if you are talking about generics. But RTL and FMX should not use ARCed instances anyway.

And for user code, kindly let's us decide for ourselves. Again, non ARC instance pointer would be able to hold ARCed instance

Quote

This would end up as an extremely complex scenario.

For whom, may I ask? For me this is Merlin's blessing!

Quote

ARC has a cost on objects lifetime and object interaction (unless you cautiously use const for all parameters passing). A positive effect of disabling ARC is a measurable performance improvement

lame, lame excuse. Let us disable ref-counting on strings too, surely rookie still can't grasp that strings, dyn-arrays, and interfaces should always be cast with const. And why use cautiously , you should use them boldly. What is the scenario when const  can backfire, pray I tell? Surely, if compiler could detect unused variables, it also can omit addref/release + exception frame generation, when said argument is not assigned?

---

Had your company 1 half-bad compiler engineer, even completely unfamiliar with the source code, on-demand  ARC would be implemented in 1 month.

 

Surprisingly, you don't need to understand source code completely to be able to successfully modify it. Find places in syntaxer ( can't believe top-down parser may be that complex), add auto class notation, auto constructor invocation. Make auto class type distinctive  from normal, in the symtable. Make sure that VMT gen would process auto and normal class twice.  Find places of interest : class instantiation, assignments, by-ref arg passing. Make sure those assignments either ARC to ARC, Normal to Normal , or unsafe(ARC) to Normal. Do your patch. Cover with test (something your guys don't usually do) Fix issues.

--

Job done.

Edited April 14, 2019 by AlekXL
typos

[AlekXL 8]

49 minutes ago, Dalija Prasnikar said:

I am critical, because you are seeking collaboration and your attitude is making more harm than good.

Your way isn't working. We won't live that long, to encounter a sunny day, with such pace.

4 hours ago, Dalija Prasnikar said:

introducing 8bit strings to all platforms. If nothing else, this makes me hopeful.

Just listen to yourself! They did disable 1-byte strings on Android platform, which internally use Utf8 encoding. Extremely stupid move. Then they reformed. That happened long time ago, I don't even remember when, but you still hopeful.

 

Now, let me try my way and get dirty for others' sake. Maybe I'm not the guy Delphi need, but surely the one it deserves.

53 minutes ago, Dalija Prasnikar said:

 Not just for your particular request, but it is undermining other efforts in similar areas.

Vague. I am self-proclaimed Delphi evangelist, and those who fall to heresy, would not get my approval.

[David Heffernan 2345]

You've really raised an army here Alek

Edited April 14, 2019 by David Heffernan

[Daniel 417]

Okay, we will stop the discussion here.

You are now discussing so many different topics here that this discussion will not come to a meaningful end.

Nearly each and every technical aspect from the Delphi-language can be seen as "somewhat related to the compiler" but this is not the way we define topics here. This might have worked more or less in the old nntp-usenet, but not in a linear web-based forum.

So we really have to stop here unless there are really new aspects about open-sourcing the compiler (e.g. the announcement from Embarcadero that they did this...)

 

//edit: Feel free to open another thread to discuss a specific technical topic like ARC or AnsiStrings or ... but please do not squeeze different discussions into a single topic here.

[Rudy Velthuis 91]

On 4/8/2019 at 9:15 PM, AlekXL said:

Learn to read

I see you are an expert in making friends. <g>

[Rudy Velthuis 91]

On 4/14/2019 at 6:30 PM, AlekXL said:

But why would he say, that on-demand arc is not viable?

Quote

but an ARC-enabled object won’t easily coexist with a non-ARC-enabled one

they would coexist just fine.

No, they would not.  Note one bit. The only objects that could co-exist would be manually and automatically refcounted objects. But normal objects are not refcounted, in Delphi (unlike in macOS and iOS).

 

So again: no, they would not easily coexist at all!

[Remy Lebeau 1393]

On 4/14/2019 at 9:59 AM, Daniel said:

Okay, we will stop the discussion here.

Can you lock this thread from further posts?

[Daniel 417]

Technically yes - but I’d prefer a common agreement to slow down here.

[Dalija Prasnikar 1396]

My comment under the QP request that explains my stance on the matter. I don't want to enter into endless discussion there because it is ultimately the wrong place.

 

@Marco Cantu enabling access to compiler (or any other sources) and allowing community contributions does not mean giving it for free. You can put up any license terms you want. I know that enforcing license is harder than having closed source, but people that will violate such licenses will just use cracked versions anyway - that is far simpler that rebuilding compiler on their own, only handful of developers would go that route.

While Delphi compiler code might be worth studying for those interested in compilers, I highly doubt there is any ground breaking technology there worth stealing nowadays. From that perspective opening compiler (or other sources) does not seem like showstopper.

 

@All however, open sourcing (I will use that term, regardless of what the actual licensing model would be) is not as simple as mere dumping source on GitHub. Somebody (Embarcadero employee), extremely knowledgeable in particular codebase would have to be gatekeeper - managing pull requests and communicating with community in general. 

 

In order to open sourcing to be successful endeavor there must be significant number of contributions made, without that - the whole thing makes no sense for Embarcadero (and for the community as well - beyond satisfying everyone's curiosity). In that case gatekeeper(s) will have their hands full. Not to mention that for every successful contribution, there will be countess others that will have to be reviewed and rejected for various reasons. That part introduces additional cost to Embarcadero and ability to develop code on their own, because part of the resources that would be used to developing in house will have to be spent for community interaction. 

 

Eventually, gains from community code can outweigh the costs, but that would take some time. Also, in code base that large, it may take years for community to start contributing in meaningful manner. And gate keeping would have to start immediately. There will be developers that will jump in from the start trying to fix their pet pevee without looking at bigger picture (breaking things like bull in a china shop).

Now, that might look like - it it would take long time, let's start early then... but like I said, initial cost in terms of resources might be too high. Especially, now when there is a lot of work to do on various compilers - new ones (Android and macOS 64bit), including introducing some new features that are already in the works.

 

From that perspective, open sourcing compiler in general... yes, but I don't think now (or any time soon) would be the right time.

Right now (or soon enough), opening up sources to RTL/VCL/FMX would be much better move - we already have the sources, we already know our way around them. Community at large can more successfully contribute to those sources than to the compiler. Again, that kind of project would also have some initial cost, but I believe it could pay off much sooner.

 

Open sourcing is the double edged sword - in order to be successful it takes resources that would otherwise be spent on in house development (with much higher initial progress rate). But the push for open sourcing (because, of the illusion that community can fix things faster and better) is greater in times when internal resources are spread thin, than in the times when everything works peachy. So it is kind of damned if you do damned if you don't situation.

[Sherlock 663]

Very insightful and concise @Dalija Prasnikar. I would even go so far as to say: The compiler is not the issue, it's the libs (RTL/VCL/FMX) as you said, but also the IDE that might profit of opening up, and still there would have to be a team of code reviewers that would do nothing but prevent crap faulty/low quality code from getting into the base. Not sure if Embarcadero would want to make that investment.

[Markus Kinzler 174]

Quote

The compiler is not the issue, it's the libs (RTL/VCL/FMX) as you said

The sources of the libraries are already "visible" for customers.

[Sherlock 663]

I know that, but how many successful pull requests have you gotten into RTL/VCL/FMX so far?

[Dalija Prasnikar 1396]

2 hours ago, Markus Kinzler said:

The sources of the libraries are already "visible" for customers.

Yes, but sometimes it takes several iterations for some fix, or needed feature to end up in release. And patching on your own takes time. If everyone has to do it that is huge waste of time. 

[dummzeuch 1505]

It could be an option to have a (restricted access) repository somewere with the RTL/VCL code where "the community" collects patches for everybody to use.

But that poses several questions:

• Which versions of the RTL/VCL are maintained there? Only ever the latest one? But what about people who are stuck with older versions for whatever reason?
• What about forward- and backporting fixes if multple RTL/VCL versions are maintained?
• Who maintains it? This/ese person(s) would be the gate keeper(s) to proposed patches.
• Who grants / revokes access and on which criteria? Is every Delphi customer allowed access, even if he only ever bought Delphi 1? Or is it restricted to the version(s) he bought? How does he prove that?
• Even if only the latest RTL/VCL is maintained, what about the .x.y Delphi releases? These are of lately only available to customers with subscription.

I think that could be quickly become a nightmare for the maintainer(s), not just the technical issues, but even more the legal issues.

[Joseph MItzen 251]

3 hours ago, Sherlock said:

there would have to be a team of code reviewers that would do nothing but prevent crap faulty/low quality code from getting into the base.

I would gladly forego the open sourcing if we could just get such a team now.

[AlekXL 8]

On 4/16/2019 at 8:53 PM, Rudy Velthuis said:

No, they would not.  Note one bit. The only objects that could co-exist would be manually and automatically refcounted objects. But normal objects are not refcounted, in Delphi (unlike in macOS and iOS). 

  

So again: no, they would not easily coexist at all!

I explained in detail, how such objects can coexist. Learn to read.

If you so sure, pray tell,

{$J-}
const strTypedConst:string='FOO';
var   bar:string;
begin
bar:=strTypedConst
end.

Is strTypedConst  constant is ref-counted? Effectively? And bar variable? What actually happens, when its value assigned to bar?

Same goes to non-arc objects -- they just have their FRefCount locked , by some negative "magic" value, in constructor. Effectively, their are not ref-counted.

 

Surely you know, that ctor invocation has hidden param, which tells whether to call NewInstance, or not (1 or 0 respectively). Why not add value 2, which would instruct the ctor, which memory model to use (1 -- for classic objects FRefCount locked, 2-- for auto   objects.

 

And yes, before you ask stpd question -- assigning non ref-counted instance (variable) to a ref-counted variable should be forbidden at compile time -- I wrote this before.

8 hours ago, Dalija Prasnikar said:

Somebody (Embarcadero employee), extremely knowledgeable in particular codebase would have to be gatekeeper - managing pull requests and communicating with community in general

So maintaining 1 expert "gatekeeper" is problem to EMBT? Some Russian or Chinese guy, who wouldn't ask more that 2-3 grand a month, working remotedly? If yes, then EMBT should close their RAD business, not wasting our time.

And before you ask, the gatekeeper would have time to catch up, before pull request would become non-stop, and contributors would teach him basics for free.

8 hours ago, Dalija Prasnikar said:

From that perspective, open sourcing compiler in general... yes, but I don't think now (or any time soon) would be the right time.

There is no right day like today, there is no right time like now. That should have been done years before. Window of opportunity might close very soon.

8 hours ago, Dalija Prasnikar said:

Right now (or soon enough), opening up sources to RTL/VCL/FMX would be much better move

System.pas is part of RTL, good luck compiling it with wrong compiler version! Have you ever dare to recompile it?

As for FMX, old RAD versions users would love to have FMX bugs fixed for free, and damn the EULA.. Who would delve into which version of FMX is actually compiled into your app?

Your advice would ruin the company. Well done.

[AlekXL 8]

5 hours ago, dummzeuch said:

I think that could be quickly become a nightmare for the maintainer(s), not just the technical issues, but even more the legal issues.

At least somebody thinks 1 step ahead here.

Edited April 17, 2019 by AlekXL
typos

[Rudy Velthuis 91]

3 hours ago, AlekXL said:

Learn to read.

You are really an expert in convincing others to support your ideas. Using a tone like that is certain to win them over. <g>

 

I did not read the entire topic (7 pages an counting...) and I will not. If you had anything useful to say, then repeat it.

 

But no matter what you wrote, I'll write it again: no, they won't easily coexist at all.

Edited April 18, 2019 by Rudy Velthuis

[Rudy Velthuis 91]

3 hours ago, AlekXL said:

Same goes to non-arc objects -- they just have their FRefCount locked , by some negative "magic" value, in constructor. Effectively, their are not ref-counted.

Heh, that doesn't even work properly for interfaces, even if they are not refcounted, You can make objects not being refcounted, but they would still live in an ARC environment, with al the possible problems and disadvantages. Note that your example contained a string constant, not just a non-refcounted object. Obviously you thought about it, but didn't think this through to the bitter end.

 

So again: no they will not coexist easily.

[AlekXL 8]

3 hours ago, Rudy Velthuis said:

Note that your example contained a string constant, not just a non-refcounted object

typed string constants do have tstrrec structure, their values can be stored in string variables without copying its value, and magic functions to addref and release are called on them, with no effect. So they live in the world of arc without their lifetime being affected by this.

That's how non refcounted instances should live

 

7 hours ago, AlekXL said:

bar:=strTypedConst

what's refcount value of instance pointed by bar, after this assignment, in your opinion?  

Edited April 18, 2019 by AlekXL