Delphi compiler need to be opensourced

[AlekXL 8]

Guess I need to repeat myself

On 4/14/2019 at 6:16 AM, AlekXL said:

We need to have an option to reference count just for explicitly designated object types and variables. And a first-class one, without need to build an interface wrapper.

Which means not some crappy wrappers like shared_ptr, no, lets introduce compiler magic and (non-breaking) language  changes.

like

FObject: auto TObject; // compiler, just manage the lifetime automacally

type TMyRefObject = class auto (TObject) // all instances are automatic

var manualRef := unsafe(RefCountedObject); // Compiler, I understand the risk, just give me raw pointer

var autoInline := auto TObject.Create;// compiler, manage it automatically

 

FRefCount should be stored at negative offset (I think, like in strings) for all instances -- ref-counted or not. But if a variable is not "auto class" _Addref and _Release should not be generated, on copy or de-refence.

Notable exception -- self special variable, with should call those in both cases. That way the each class implementation code would work for both cases, allowing auto class to be derived from non-refcounted one.

Magical _Addref and _Release would have no effect, if FRefCnt is negative -- this is a fail-safe during runtime. FRefcount should initially set during constructor invocation, based on hidden param, either to negative, or to 0 (+1, before ctor starts), (1 -1= 0, after constructor is done)

 

 

 

[AlekXL 8]

16 hours ago, Dalija Prasnikar said:

Eventually, gains from community code can outweigh the costs, but that would take some time. Also, in code base that large, it may take years for community to start contributing in meaningful manner. And gate keeping would have to start immediately. There will be developers that will jump in from the start trying to fix their pet pevee without looking at bigger picture (breaking things like bull in a china shop).

Now, that might look like - it it would take long time, let's start early then... but like I said, initial cost in terms of resources might be too high. Especially, now when there is a lot of work to do on various compilers - new ones (Android and macOS 64bit), including introducing some new features that are already in the works.

  

From that perspective, open sourcing compiler in general... yes, but I don't think now (or any time soon) would be the right time.

(Emphases are mine) Not-so-thinly-veiled sabotage of my motion! Too long, too expensive, too bad, too expensive again, too soon...

After some time, I have no doubt you'll politely and wisely concede: too late

Edited April 18, 2019 by AlekXL

[Dalija Prasnikar 1396]

11 hours ago, AlekXL said:

So maintaining 1 expert "gatekeeper" is problem to EMBT? Some Russian or Chinese guy, who wouldn't ask more that 2-3 grand a month, working remotedly? If yes, then EMBT should close their RAD business, not wasting our time.

And before you ask, the gatekeeper would have time to catch up, before pull request would become non-stop, and contributors would teach him basics for free.

 

 

Seriously???? You would leave five year old alone in control of the house when parents are away? Actually, it would be more like leaving the five year old with matches and can of gasoline.

 

Let me say it again: Somebody (Embarcadero employee), extremely knowledgeable in particular code base would have to be gatekeeper - managing pull requests and communicating with community in general.

 

That somebody would have to be top gun compiler engineer, someone from existing team who knows code base well, not someone who would they just throw in working for peanuts. So yes, under current circumstances that would take valuable resources away from current compiler development. And that is something we (or Embarcadero) cannot afford at this time. 

 

Yes, it would be good if Embarcadero would expand their core teams. That would be good move regardless of open sourcing . But even if they manage to hire good engineers today, it would still take quite some time before such expanded team could bear the burden of open sourcing.

 

 

11 hours ago, AlekXL said:

System.pas is part of RTL, good luck compiling it with wrong compiler version! Have you ever dare to recompile it?

As for FMX, old RAD versions users would love to have FMX bugs fixed for free, and damn the EULA.. Who would delve into which version of FMX is actually compiled into your app?

Your advice would ruin the company. Well done.

LOL

 

And leaving amateurs to rampage all over the compiler, yeah, that would end really well.

 

As far as RTL/VCL/FMX is concerned, it is not about catering for all old versions in existence out there. It is about going forward, so no compiler version would not pose a problem.

[Dalija Prasnikar 1396]

4 hours ago, AlekXL said:

(Emphases are mine) Not-so-thinly-veiled sabotage of my motion! Too long, too expensive, too bad, too expensive again, too soon...

After some time, I have no doubt you'll politely and wisely concede: too late

I am not sabotaging anything. I am merely presenting the facts that must be taken into account. Without that it may look like open sourcing the compiler is just one GitHub click away and Embarcadero is not doing that "good" move on purpose because <insert conspiracy theory du jpur>

Also, when it comes to sabotaging, like I already said, you don't need any help in that area, you are perfectly doing that yourself. 

 

Your request lacks any real arguments and is basically insult to everyone working at Embarcadero.

[Jacek Laskowski 57]

Probably Alek definitely simplifies and exaggerates with ease the entire process of opening and managing sources. 

However, one thing is certain. Delphi requires quick and bold changes, or else it will drown.

[Markus Kinzler 174]

EMBT needs a long-term strategy. This could include a open source part.  (Only) Open Sourceing Delphi (or parts) isn't the solution, it can be the end.

[Dalija Prasnikar 1396]

16 hours ago, dummzeuch said:

It could be an option to have a (restricted access) repository somewere with the RTL/VCL code where "the community" collects patches for everybody to use.

But that poses several questions:

• Which versions of the RTL/VCL are maintained there? Only ever the latest one? But what about people who are stuck with older versions for whatever reason?
• What about forward- and backporting fixes if multple RTL/VCL versions are maintained?
• Who maintains it? This/ese person(s) would be the gate keeper(s) to proposed patches.
• Who grants / revokes access and on which criteria? Is every Delphi customer allowed access, even if he only ever bought Delphi 1? Or is it restricted to the version(s) he bought? How does he prove that?
• Even if only the latest RTL/VCL is maintained, what about the .x.y Delphi releases? These are of lately only available to customers with subscription.

I think that could be quickly become a nightmare for the maintainer(s), not just the technical issues, but even more the legal issues.

Valid questions...

 

• Embarcadero would have to appoint gatekeepers - probably their employees, although here they could use knowledgeable community members, but again those community members would have to report back to Embarcadero.
• For every point release maintaining interface compatibility is imperative. So there would have to be stable interface compatible version branch, and unstable interface breaking branch for future point release. 
• From previous points, it is clear that only current release would be maintained. Any back-porting would have to be done by Embarcadero respecting older release interface compatibility. Probably very few (and more critical patches) would be applied to older versions and only few releases back (maybe even just one - depending on the point release frequency)
• Access rights - once you put code out it is hard to revoke access, and maybe it would make little sense for doing so. As far as initial access rights are concerned - while it would be nice if everyone could have access, realistically it only makes sense to give access to people that have current (or one older) release. You have to have means to contribute and if you are sitting on Delphi version that dates 5 years ago, you cannot contribute in meaningful manner to current and future releases.

 

I know that people would love to have fixes for older versions, but I don't think amount of effort needed for such back porting would be sustainable. At least not for Embarcadero. What community in general could do in that regard - I don't want to speculate (including legal reasoning) At the end even now, we all have the sources, and nothing prevents people to organize some dark web code patching.

[Микола Петрівський 10]

6 hours ago, AlekXL said:

typed string constants do have tstrrec structure, their values can be stored in string variables without copying its value, and magic functions to addref and release are called on them, with no effect. So they live in the world of arc without their lifetime being affected by this.

That's how non refcounted instances should live

 

what's refcount value of instance pointed by bar, after this assignment, in your opinion?  

No matter what you do, sooner or later RTL will have to insert a call to UniqueString. For example when you pass your variable to a function with var parameter. For strings this is not a problem, because they are value types, their copying does not have side effects. For objects you can't guarantee, that copying is safe. You can ask programmers to mark their objects as copy safe, and add additional compiler restrictions for unsafe objects. But how many times some object will happen to have wrong mark, and how easy it will be to debug and fix it?

[AlekXL 8]

21 minutes ago, Микола Петрівський said:

No matter what you do, sooner or later RTL will have to insert a call to UniqueString

What are talking about? Uniquestring is about COW on top of ARC! Nothing to do with ARC itself. Where I suggested to make instances COW capable? Think!

If  ARC pointer to instance is to be passed as var parameter, it's just passed without anything.

procedure Modify(var instance:auto TObject);// auto here is mandatory to accept ref_counted instances
begin
	instance := auto TObject.Create();//instance is replaced, _Release is called, then ctor!
    // _Release with do NOTHING if FRefCount is NEGATIVE!
end;

procedure Test();
begin
	var inst := auto TObject.Create();
    Modify(inst);//just pass without any fuss
    var classic = TObject.Create;// Non refcount, FREFCOUNT := -1000
    Modify(classic);//just pass without any fuss
end;

 

[AlekXL 8]

2 hours ago, Dalija Prasnikar said:

Seriously???? You would leave five year old alone in control of the house when parents are away? Actually, it would be more like leaving the five year old with matches and can of gasoline. 

  

Let me say it again: Somebody (Embarcadero employee), extremely knowledgeable in particular code base would have to be gatekeeper - managing pull requests and communicating with community in general. 

You don't realize, what a monster you could hire for 2-3 grand in Russia, do you? C++ expert with 15y+ experience, and parsing expert on top of that. He would run circles around both me and you. Need I repeat myself?

Quote

Surprisingly, you don't need to understand source code completely to be able to successfully modify it.

And if the compiler source code is unmanageable even with such expert, well, it's beyond salvation then, and there is no reason to hide junk from community. Better start reworking it now, or Delphi is doomed.

2 hours ago, Jacek Laskowski said:

However, one thing is certain. Delphi requires quick and bold changes, or else it will drown. 

Absolutely! Bold moves made Delphi great. Bold and clever, not bold and stupid.

2 hours ago, Markus Kinzler said:

EMBT needs a long-term strategy. This could include a open source part. 

You mean, except milking Delphi user-base to the end? We both know what they need. Real feedback system, accountability, responsibility ... Marco is clearly failing in that department. Simply out of the loop.

2 hours ago, Markus Kinzler said:

(Only) Open Sourceing Delphi (or parts) isn't the solution, it can be the end.

The end is looming anyway. We are loosing young developers, to python, C#, etc. Even FPC/Lazarus somehow is getting some momentum

2 hours ago, Dalija Prasnikar said:

Embarcadero would have to appoint gatekeepers

sure, 1 gatekeeper for compiler is too expsensive, and several ones for RTL/VCL/FMX are quite affordable, aren't they? Your coherence is exemplary  (of you-know-what)

2 hours ago, Dalija Prasnikar said:

As far as initial access rights are concerned - while it would be nice if everyone could have access, realistically it only makes sense to give access to people that have current (or one older) release.

You don't get it, how EMBT makes money, do you? They charge users for bugfixes, especially in FMX, forcing to buy new versions or pay for subscription.

Once people would able to fix current version of the libraries, by joint effort, or Merlin forbid, would have access to a newer release, sales will drown.

Edited April 18, 2019 by AlekXL
clarication

[Sherlock 663]

@AlekXL Please consider reading your posts while taking your opponents point of view. Then please consider them to be human beings. Then alter your text, defusing the disses, then take a deep breath, read it again, insert some final niceties and finally post it. Moderators have been lenient thus far, but patience is growing short.

[Rudy Velthuis 91]

8 hours ago, AlekXL said:

typed string constants do have tstrrec structure, their values can be stored in string variables without copying its value, and magic functions to addref and release are called on them, with no effect. So they live in the world of arc without their lifetime being affected by this. That's how non refcounted instances should live

They are not **objects** that can actively reference **other** objects. See how this doesn't even work for interfaces. What makes you think it would be different for objects? Again: no, they can not easily live together

8 hours ago, AlekXL said:

 

what's refcount value of instance pointed by bar, after this assignment, in your opinion?  

What bar? Are you talking about a passive string again?

[AlekXL 8]

1 minute ago, Rudy Velthuis said:

They are not **objects** that can actively reference **other** objects. See how this doesn't even work for interfaces.

Please give an code example, not a vague words. What is the mysterious problem you taking :  "this isn't working for interfaces"?!

Can non-refcounted instance hold member instance of interface? Yes. They do that all the time. Can an ref-counted instance hold a reference to non-ref-counted object? Yes, absolutely.

7 minutes ago, Rudy Velthuis said:

What bar? Are you talking about a passive string again?

string stored in executable image, readonly, with magic FRefCount value (negative)

[Markus Kinzler 174]

Please stop discussing off-topic here! You're free to open futher threads for the (interessting) topics!

[AlekXL 8]

4 minutes ago, Sherlock said:

Please consider reading your posts while taking your opponents point of view.

I strive to do so, every time. Are my comments imply I don't understand their position? Please supply an example or two

6 minutes ago, Sherlock said:

Then please consider them to be human beings.

All humans make mistakes. But some humans' mistakes would cost too much, if those deeds are not condemned

23 minutes ago, Sherlock said:

Moderators have been lenient thus far, but patience is growing short

Look, I tried my best to never give negative judgement on people, just their doing or words. However, if you think my effort is not enough, I'll try to be more subtle with my assessments

[Sherlock 663]

37 minutes ago, AlekXL said:

Look, I tried my best to never give negative judgement on people, just their doing or words. However, if you think my effort is not enough, I'll try to be more subtle with my assessments

Thank you, that is all we're asking.

[AlekXL 8]

Well, see what Marco replied in QP

Quote

I agree time on compiler quality needs to remain very high, and it is right now.

Delphi compiler is of high quality right now .. Maybe it's me, who is delusional?

[Sherlock 663]

Are you sure he is talking about the quality...or rather about the time invested? And please, words like delusional are not considered to be kind or respectful.

[Rudy Velthuis 91]

9 hours ago, AlekXL said:

(Emphases are mine) Not-so-thinly-veiled sabotage of my motion! Too long, too expensive, too bad, too expensive again, too soon...

After some time, I have no doubt you'll politely and wisely concede: too late

If it were so desirable and/or so easy to maintain the Delphi compiler, I wonder why FPC didn't make Delphi obsolete already.

 

I guess because after all there are not so many people capable of maintaining, nor enough people inclined to do it. I also gues that the task is not so easy.

 

I have said it before, and I'll say it again: I use a lot of open source and I love open source and I have participated in open source, but the notion of "open source it so you can always fix it yourself" is pure nonsense. Even if I suffer from some bugs (and indeed, Rio seems to have quite some new bugs and slowdowns caused by the -- incompletely -- removed mrecords, i.e. the runtime changes still affect current code, even if mrecords themselves were removed again), I am not capable nor willing to read the entire (possibly hard-to-read) source code of the entire compiler and then I'm not capable of making a decision what I can safely change without causing a multitude of other bugs. I will leave that to others. I am not so sure there are enough "others" who could or are willing to maintain the Delphi compiler. Nor am I willing to hire someone to fix them for me. People who could do that would probably require large fees too.

 

So no, open sourcing the compiler is useless, IMO. Rather lean on Embarcadero and confront them with "love withdrawal" or other sticks behind your back and let them do it. I agree we need bug fixes and new features. But open sourcing won't provide them. Only pressue on Embarcadero can.

Edited April 18, 2019 by Rudy Velthuis

[AlekXL 8]

i guess i misunderstood. Still Marco practically said that effort on the compiler is the same as before. Nothing is to be changed, in his opinion. 

 

Surely, "delusional" was about myself 

[Rudy Velthuis 91]

Just now, AlekXL said:

i guess i misunderstood. Still Marco practically said that effort on the compiler is the same as before. Nothing is to be changed, in his opinion. 

 

Surely, "delusional" was about myself 

I have absolutely no idea what Marco said. Doesn't change what I wrote.

 

I do think that the efforts to improve and change the compiler have not diminished, although it doesn't always run very smooth (shit happens). Could that be what he meant?

 

FWIW, probably off-topic, but I do also think that the way mrecords were (IMO hastily) implemented was wrong and not fixable in the time period they had. But the removal was also a little too hasty, and many of the code generation changes they deemed necessary (they did; I didn't always) were not completely removed and still affect the current state of affairs. They should, IMO, implement managed records properly, not only so they could create shared pointers and the like (that can be done with the hlp of interfaces already, see Barry Kelly's blof posts and things like Spring4D).

 

No, that is not the reason I want them. Mrecords would give us an opportunity to greatly tune our records by avoiding the slow runtime typeinfo/RTTI based runtime processing and replace it with manual compile-time tuning. But that is off-topic already.

[AlekXL 8]

6 minutes ago, Rudy Velthuis said:

 it were so desirable and/or so easy to maintain the Delphi compiler, I wonder why FPC didn't make Delphi obsolete already.

Because 

1. they support too many platforms

2. do not pursue delphi compatibility strong enough

3. are lacking in debugger and ide department, which prevents synergy and positive feedback

still right now FPC is getting critical mass… i believe

 

[AlekXL 8]

21 minutes ago, Rudy Velthuis said:

I guess because after all there are not so many people capable of maintaining, nor enough people inclined to do it. I also gues that the task is not so easy.

Here are 2 cases, why Delphi might be in not-so-good condition, to put it politely

1. EMBT is for "cost optimization"  to the extreme.. Compiler engineers' team is lacking manpower, or they are mostly too cheap. Their resources stretched thin. This means Delphi is doomed (unless compiler is GPLed), and nothing can be done about it, until it's too late. Opensourcing the compiler then would just create some fail-safe..

2. Compiler engineers' team is too expensive. Ask too much and do too little. In that case opensourcing the compiler could bring new, knowledgeable team of really devoted developers, who would compete each over, and would ask less money. No, they wouldn't be many. But even 5-10 of those around the globe would make all the difference. Marco wouldn't  want accept fixes for free? Fine! Pay for a fix! Give some $50-$100 to poor student (some of them are incredibly bright), and own his work.

[Rudy Velthuis 91]

2 minutes ago, AlekXL said:

This means Delphi is doomed

Ah, yes, once again. <sigh>

 

Delphi has been called doomed, or dying, or on the way out for almost as long as it exists.

 

And yet it is still going strong despite the current problems. Yes, it has issues, and I wish this was not one of those darned problematic versions that pop up every now and then, but calling it doomed is hyperbole, IMO.

 

FWIW,

Frazer: We're doomed! We're all doomed! Doomed I tellya!

Mainwaring: Frazer, stop rolling your eyes like that!

[Rudy Velthuis 91]

13 minutes ago, AlekXL said:

Compiler engineers' team is too expensive

And you think that unpaid open source volunteers would do a better job because...? Or, if someone were to be hired by a group of Delphi enthusiasts, what makes you think he/she/they would be cheaper or better?

Edited April 18, 2019 by Rudy Velthuis