Delphi compiler need to be opensourced

[AlekXL 8]

4 hours ago, Ugochukwu Mmaduekwe said:

Using C++ libraries was ok at a time when Delphi was Windows only but please we are in the Cross Platform era now.  

 

  

And please do note that I am referring to free OpenSource maintained libraries not closed source sharewares.

Well, honestly I can't claim I am expert in linking C libraries to Delphi or FPC on non-Win platforms, but surely it's not that bad? I mean, yes, you need some effort to do this, but considering this is one-time thing, why this is trouble?

 

I honestly don't understand the effort implementing such heavy task on python, javascript or even Java. Probably the very reason such libraries is paid in Delphi is the practical approach. Try to sell python software, or javascript.. Delphi app can be sold..

 

And perhaps we need to create a C-to-Delphi conversion tool, just to make such import easier.

[David Heffernan 2345]

6 hours ago, Ugochukwu Mmaduekwe said:

Using C++ libraries was ok at a time when Delphi was Windows only but please we are in the Cross Platform era now. 

Which platform doesn't have good C and C++ compilers? 

[AlekXL 8]

3 minutes ago, David Heffernan said:

Which platform doesn't have good C and C++ compilers? 

probably problem is linking OBJ to a unit and getting unresolved symbols, which is from depths of C/C++ RTL.. Then you need to un-archive libraries/lib to a object files, and try to feed those needed to Delphi/FPC making it happy. Tedious, but not impossible..

If one getting paid for development, he just should brace himself get job done. Otherwise let him play with toys like python.

[AlekXL 8]

On 4/11/2019 at 7:51 AM, Joseph MItzen said:

Meanwhile, the FreePascal crowd has vowed never to accept a patch for type inference

Please, supply a proof of that, because someone says it's just a rumor. Citation and where you get this would be helpful.

Meanwhile, we are on page 4, and this topic about opensourcing Delphi compiler, please vote on QP , if you didn't already

Additionally consider supporting Delphi-compatibility issue on Newpascal

https://github.com/newpascal/newpascal/issues/6

[Daniel 417]

We are on page #4 because of a lot of off-topic.

Also issues of newpascal are a separate topic - might be off topic also in a delphi-related community like this.

And I will clearly not vote for this request.

[AlekXL 8]

1 minute ago, Daniel said:

Also issues of newpascal are a separate topic - might be off topic also in a delphi-related community like this

So, are you suggesting to start a new topic on FPC/Newpascal fork?

For me, those topics are closely related, since actually some closed-sourced apps were open-sourced due to threat from already free competitors.

And surely, FPC can't be off-topic in  Delphi community.

[Markus Kinzler 174]

Quote

So, are you suggesting to start a new topic on FPC/Newpascal fork?

Yes we do.

[Stefan Glienke 2002]

If this thread is only a small foreshadowing of how the community (or rather individuals) would react and respond to an open source project and how things are getting discussed I have to say that I would not want to be part of it.

Yes, everyone is permitted to his own opinions but the how is important - and I can see in other communities like for example the C# compiler or .Net Core projects that developers don't want to deal with bs and toxic waste but communicate in a civilized manner.

 

... and that coming from me known as someone who can voice his opinions in a strong way garnished with cursing often enough ...

[Lars Fosdal 1792]

I am totally for expanding FPC to cover everything Delphi can do and more. I would welcome it and applaud it. I don't think EMBT would mind, either.

 

Giving up their own compiler is completely different thing.

 

However, I don't believe FPC will ever catch up. Look at attributes which are still not supported. My Delphi code won't even compile on FPC, and that is not a EMBT problem. 

[Remy Lebeau 1394]

15 hours ago, AlekXL said:

Perhaps, Embarcadero shouldn't reinvent the wheel , but push VCL to all platforms.. Something like CrossVCL, but also for mobile.

Borland tried that once before.  The CLX framework in Delphi 6 was a cross-platform version of the VCL.  Most of the interfaces remained the same, just the internal implementations changed.  That didn't work out so well for them, CLX had to be scrapped.

[David Heffernan 2345]

8 minutes ago, Remy Lebeau said:

Borland tried that once before.  The CLX framework in Delphi 6 was a cross-platform version of the VCL.  Most of the interfaces remained the same, just the internal implementations changed.  That didn't work out so well for them, CLX had to be scrapped.

CLX was actually alright. I built a cross platform app with it back in the day and lived for quite some time

[David Heffernan 2345]

1 hour ago, Lars Fosdal said:

Giving up their own compiler is completely different thing.

Open source isn't giving up. It's allowing contributions from outside. Emba would still be in control. 

 

The only open source project I have any involvement is Spring4d. Who do you think is the boss of that? It sure ain't me. And that's exactly how it should be.

 

A good open source project has strong management and leadership, and those contributors that can fit in and add to the development effort are facilitated. You don't just hand control to random collaborators. 

 

Stefan's point about open source collaborators is spot on. Good programmers aren't going to spend their time on open source projects unless it's rewarding and fun. I certainly could never see myself working with the OP here. 

Edited April 12, 2019 by David Heffernan

[Joseph MItzen 251]

On 4/11/2019 at 11:16 PM, AlekXL said:

1. Any company invested in developing a product written in Delphi and confronting some compiler issue would hire skilled developer to fix it.

2. Be factual, and name the libraries Delphi lacking. And also, Delphi can utilize C/C++ libraries.

 

1. We get posts here, Reddit, etc. along the lines of "Please help me install Borland Database Engine on Windows 10" all the time. A large number of the Delphi applications being maintained today are being maintained on old versions of Delphi and the companies responsible for them have no desire to spend anything to modernize them. For example, the Melissa and Doug toy company in the United States uses an in-house ERP (Enterprise Resource Planning) program written and still maintained in Delphi 7.

 

1B. Again, a "skilled developer" can't just look at one million lines of undocumented code (that might be older than they are) and fix it. It could take years for a dedicated open source community to work their way through and document that code. Even then, the only sane options would be to massively refactor and modernize it or, again more likely, simply start over.

 

1C. Who would be in charge of the code? Would Embarcadero oversee it? If so, they might not approve changes. And they're unlikely to want to give up control, as that would be giving up control of the language. Going back to Firebird, they have a poor history of properly supporting open source code, and Marc Hoffman said that one EMBT employee said during discussions "We own Pascal." Not quite the mindset required to turn over control of their compiler/language to a third party.

 

1D. If there aren't enough developers interested and able to support FreePascal or NewPascal, where are the developers going to come from to spend years trying to make sense of the old Delphi desktop compilers, a much harder task with a much longer time to payoff?

 

2. Are you serious? Get-It has about 200 packages after four years, most of which are trials. Torry's Delphi Pages, the oldest collection of Delphi software still around, has a little over 10,000 packages ( a number that's barely moved from 2012), the vast majority of which are from the Delphi 5-7 era and most of the rest being trials of paid software. In the last 30 days it's added 17 files, only one of which is for free software.

 

Now take a look here:

 

http://www.modulecounts.com/

 

Java has over 274,000 open source libraries at Maven Central, C# has over 148,000 in nuget, Python over 174,000. Java is adding 197 a day, C# 93, Python 122. Per above, Delphi added 1 in 30 days.

 

What libraries is Delphi lacking? Almost anything that doesn't involve grids (Delphi users are obsessed with grids). Serious numeric software (although you can pay $1600 for a bundle of Dew Research's matrix library, sci/stat function library, 3 basic data mining algorithms and Steema's TeeChart). Machine learning libraries. Financial libraries. Let's go smaller - I'm interested in making a media player. Delphi lacks a music file tag manipulation library (that's post-unicode and documented), audio manipulation library, replay gain library, cover art library, etc., all of which I can find on other platforms  as open source.

 

I'm working on another pet project now that involves data analysis and machine learning and if successful will require interacting with websites. I'd like to host it on a Linux Virtual Private Server eventually, which means $3500 for Delphi Enterprise. As mentioned above, the Dew Research bundle with source is $1600, bringing us up to $5100. Now let's add in Steema's TeeBi for some basic data manipulation abilities to clean and transform the data, which is $279. Wait, the only plugin it works with out of the box to export to Excel is TMS' FlexCel. So unless we want to find an open source Delphi Excel library and integrate it ourselves, that's another $141USD. Now we're up to $5520. A capable profiler? Smartbear AQTime Pro - another $600. Now we're at $6120. Let's toss in HelpNDoc for documentation for another $280USD. Now we're at $6400 so far. I want some neural networks... Boris Mitov has 1980's-era Kohonen self-organizing feature maps and basic backpropagation for $389 without source or $1369 with source. That's madness; I coded both algorithms circa 1994 for a school project in Turbo Pascal. Let's use FANN instead, which is an open source library that implements some improvements to basic backpropagation and has a fast C core. There's also an open source implementation of the NEAT neural algorithm for Delphi, but not the next-gen HyperNEAT. Oh well. Genetic programming? RiverSoftAVG has a decent library with basic features for $150. So $6550 so far and we haven't looked at logging or databases. We'll need to scrape and parse HTML - DIHtmlParser is $123 with source code but is very low-level. There's a higher-level parser on Github, but the only documentation is in Japanese. There's an "HTML Component Library" available that seems rather nice, but it's bundled with a lot I wouldn't need, raising the price to $350. Come to think of it, once all of this comes together and is working I'm going to need a high-level process to manage all of the various software tasks - downloading data, parsing data, processing data, interacting with the right websites at the right time, not launching a process that depends on another that failed, etc. Sadly, there simply aren't any "pipeline " libraries for Delphi at any price.

 

And this is the point when one does the math and realizes they could take that money and buy a new desktop with a 32 core AMD Threadripper, 64GB memory, high-end graphics card for GPU acceleration, 3 NVMe SSDs in RAID 0 for hyper-fast caching with a few large-capacity hard drives in RAID 5 as backing storage, large curved 4K monitor,  etc. and even have some money left over. Seriously. And they can use the Python scientific software stack of NumPy, SciPy, Matplotlib for matrices and graphics, but also use modern "grammar of graphics" (functional composition) libraries like plotnine or altair, something TeeChart doesn't offer. Pandas has had almost a 10 year lead time on TeeBI and is far fuller featured and much better documented. Scikit-learn offers a massive machine learning suite, unlike Dew Research's anemic three methods and no support functions.  NeuPy offers a vast amount of neural networks for free, plus one can download and use TensorFlow from Google or PyTorch from Facebook and gain support for modern deep learning frameworks that do things like power self-driving cars. Let's see, there's a HyperNEAT implementation too; and genetic programming libraries like DEAP which can scale across all the machines on a network if you wish, and also the best clustering algorithm, DBSCAN, which I couldn't find for Delphi. There's the well-known scrapy framework, as well as BeautifulSoup for high-level HTML parsing. Pipeline programs? Companies like Spotify have open-sourced tools like luigi and AirBNB has airflow. Both can produce web pages to monitor the performance of your processes and view dependency graphs in real time! And there's Selenium for driving the major web browsers across platforms.

 

And that's the difference between using Delphi and using a modern language with 100,000+ free and open source cross-platform libraries available. I've spent zero on software, gotten much more powerful libraries in many cases, and don't have to spend weeks or months implementing low-level libraries myself and can instead concentrate on the problem I'm trying to solve. That was the original appeal of Delphi - the VCL abstracted away the monotonous low-level Windows API to allow developers to code only the solutions relevant to their specific problem. Nowadays this is a solved problem in the form of open source software and Delphi is no longer the "rapid" application development it originally was. You're going to spend lots of time writing things that other language users can download and install from modern package managers in seconds. Your other alternative is to spend thousands of dollars on lesser libraries maintained by one person who may disappear tomorrow. In a research paper designed to discover what makes programming languages popular, the authors found from surveys that the single most important factor to developers in choosing a language is the amount of libraries available, preferably open source libraries. I've seen one developer put it this way, "Today, isn't not being popular itself a language defect?"

Edited April 13, 2019 by Joseph MItzen

[Joseph MItzen 251]

17 hours ago, AlekXL said:

2. Those who need linux, must do it themself  and test their patches not on linux, but ensure also, that those won't break 2 major platforms. Or they full-refund pay. 

3. Those who need an Apple platforms, need to contibute [and pay]. Obviously those minor platforms are targeted for gain 

 

The results of the latest Stack Overflow survey came out and show that more developers are working on OS X and Linux combined than on Windows! For all respondents, 47.5% said they primarily work on Windows while OS X was 26.8% and Linux was 25.6%. For those who said they were professional developers, it's only 45.3% Windows, 29.2% OS X, 25.3% Linux, 0.1% BSD.

 

If you marginalize Linux you're cutting off a group with high technical skills and obviously more enthusiasm for open source than your average Windows user. In fact, since Delphi's tools can't run on Linux or OS X and there's no FMX support for Linux and no 64bit support for OS X, Linux and OS X developers have more interest in FPC than the average Windows user. They need it. Treating them like Embarcadero treats them won't endear them to your project.

[Joseph MItzen 251]

17 hours ago, Schokohase said:

and I bet they already had discussed that internally (Delphi-based Delphi-compiler and open-source it). 
 

Mason Wheeler and I once had an online exchange with Allen Bauer during the end of his tenure with Embarcadero about Delphi and open source, including the chance of open sourcing any part of the toolchain. Suffice it to say that the folks at Embarcadero don't even understand open source much less are feeling ready to embrace it.

[Joseph MItzen 251]

16 hours ago, AlekXL said:

Which algorithm or hash Delphi is lacking?

well  C/C++ libs (cross)compilation  is the pain, but unlike python or java, you'll get fastest solution in the end of the day. 

As has been established elsewhere, Java is faster than Delphi, particularly with math. It blows Delphi away in SciMark benchmarks. If you were looking for the fastest solution you'd just use C/C++ itself (which is, on average, twice as fast as Delphi code, although David Heffernan has shown examples where it's even worse for Delphi). Meanwhile, it's actually far easier to interface C or C++ with Python than Delphi (particularly object-oriented C++ code) which makes it faster to actually have working code. Developer time is more expensive than CPU time.

 

I've found that jitted Python, in some small benchmark tasks I wrote, tended to match or ever-so-slightly beat FreePascal. Using Cython, which converts Python code to C++ code where possible, could yield results that were just shy of C and far faster than Delphi. Numba, a Python JIT only for mathematics, could also yield double the speed of FreePascal (which is itself often faster than Delphi on mathematical tasks but slower with strings). Numba also has the ability to automatically parallelize operations or even run code via CUDA on a GPU!

 

Another issue is that the Delphi Linux code is slow, and I haven't heard that they've ever fixed this (or acknowledged this). It's about half the speed of equivalent Delphi code, which is itself half the speed of C++. This makes both FreePascal (and even jitted Python!) twice as fast as Delphi on Linux.

[Joseph MItzen 251]

15 hours ago, AlekXL said:

https://github.com/Xor-el/CryptoLib4Pascal rings any bells?

isn't it your project? 

Wait, you thought you knew his project better than he did? Seriously?

[Joseph MItzen 251]

15 hours ago, AlekXL said:

Try to sell python software, or javascript.. Delphi app can be sold..

 

And perhaps we need to create a C-to-Delphi conversion tool, just to make such import easier.

That's a major factor that damaged Delphi, per my earlier post. People don't sell each other software libraries anymore. They open source them, which attracts more people to the language, which means more open source libraries for you to use. As I always say, "Make money with Delphi, not from Delphi". Every library that's sold only raises the barrier to entry for a potential new user. If I was being mean, I could say that Python or javascript developers don't need to sell libraries because they can find jobs. Also, their tool suites don't cost more than their desktops.

 

Now your second point gets to one of the things on my list of what I'd do if I ran Embarcadero. You're half right. We need a way to make importing easier. We don't need a conversion tool though. There's a long-running open source project whose software automatically wraps C and C++ code to make it callable from other languages! It's called SWIG, or Simplified Wrapper and Interface Generator. Referencing my point about popularity again, it even supports Modula-3 (!!!) but not Delphi. The good news is that I encountered someone on Stack Overflow whose company added support for Delphi to SWIG internally and were able to generate a fine wrapper of a complex C++ 3D graphics library with it. The bad news is that although asked his company did not appear interested in open sourcing their work. But of course, that means that it is possible to add Delphi support to SWIG.

 

Now, when I tried to broach the subject with The Powers That Be, Marco Cantu decided to deny that a problem even existed, insisting that he does not believe that the .NET ecosystem is larger than the Delphi ecosystem (!!!!). That's when I realized that if you don't acknowledge problems even exist you'll never solve them. Since he wouldn't admit there was a problem, he obviously wasn't interested in hearing my idea for solving it - in this case, I thought Embarcadero should add Delphi support to SWIG and integrate it with the IDE or include it with the installer.

 

Other languages have FFIs (Foreign Function Interfaces) that make it easy to interoperate with other languages. Delphi does almost nothing to make it easier and it's the equivalent of banging on it with a hammer to get C++ code into Delphi. Here's an example of a newer one for Python: PyBind11. (Python has a FFI in its standard library, but it's not very full-featured). But this does serve to illustrate the point that it can be even easier to interface C++ with a dynamically typed language without pointers than it often is to get C++ code working with Delphi currently.

[Joseph MItzen 251]

14 hours ago, AlekXL said:

 Otherwise let him play with toys like python. 

I'm sure you heard about the first ever photo of a black hole. The associated research paper credits the open source scientific python libraries NumPy, SciPy and Pandas. Python was also one of the tools used in the hunt for the Higgs Boson and CERN scientists have spoken at Python conferences. The PyCon conference has been sponsored by everyone from LucasFilm and Disney Studios to Hewlett-Packard, Microsoft, Canonical, Disqus, Mozilla, J.P. Morgan bank, Twitter, Hulu,  and Amazon. In fact, those were just a handful of the 2012 sponsors. Take a look at CodeRage sponsors: "Atozed Software, DevJet Software, Fast Reports Inc,  Raize Software, Scooter Software, SmartBear and Steema Software." Draw your own conclusions.

[Ugochukwu Mmaduekwe 42]

@Joseph MItzen you just analysed everything perfectly from my point of view. 👍

[AlekXL 8]

15 hours ago, Joseph MItzen said:

The results of the latest Stack Overflow survey came out and show that more developers are working on OS X and Linux combined than on Windows! For all respondents, 47.5% said they primarily work on Windows while OS X was 26.8% and Linux was 25.6%. For those who said they were professional developers, it's only 45.3% Windows, 29.2% OS X, 25.3% Linux, 0.1% BSD.

  

If you marginalize Linux you're cutting off a group with high technical skills and obviously more enthusiasm for open source than your average Windows user. In fact, since Delphi's tools can't run on Linux or OS X and there's no FMX support for Linux and no 64bit support for OS X, Linux and OS X developers have more interest in FPC than the average Windows user. They need it. Treating them like Embarcadero treats them won't endear them to your project.

Linux is important, I believe, but I'm just frustrated some useful libraries treat windows targets as 2-class citizens at best. I'm not very interested in linux currently, so I just don't want to support it myself. If Linux has many pascal developers, who'd wish to contribute to FPC, let them. I just don't want their innovations would cripple the compiler and libraries on Windows.

14 hours ago, Joseph MItzen said:

Wait, you thought you knew his project better than he did? Seriously? 

No, I realized that he was the author after posting the comment

14 hours ago, Joseph MItzen said:

I'm sure you heard about the first ever photo of a black hole. The associated research paper credits the open source scientific python libraries NumPy, SciPy and Pandas. Python was also one of the tools used in the hunt for the Higgs Boson and CERN scientists have spoken at Python conferences. The PyCon conference has been sponsored by everyone from LucasFilm and Disney Studios to Hewlett-Packard, Microsoft, Canonical, Disqus, Mozilla, J.P. Morgan bank, Twitter, Hulu,  and Amazon. In fact, those were just a handful of the 2012 sponsors. Take a look at CodeRage sponsors: "Atozed Software, DevJet Software, Fast Reports Inc,  Raize Software, Scooter Software, SmartBear and Steema Software." Draw your own conclusions.

But does Pascal even fit that ecosystem? I mean those scientists like python for its "simplicity".. Do they adopt pascal, however free and approachable it may be?

I don't understand python appeal really. How weakly typed language may be so popular? And I don't believe any software > 50K lines could be robust under those circumstances. Is't python is a script language on steroids?

15 hours ago, Joseph MItzen said:

As has been established elsewhere, Java is faster than Delphi, particularly with math. It blows Delphi away in SciMark benchmarks.

In pure math, yes, but this is memory management, Delphi/FPC is faster just because manual management is way faster than any GC. If you work with large amount of inter-related data objects, Java would not shine as bright, let alone python.

15 hours ago, Joseph MItzen said:

I've found that jitted Python, in some small benchmark tasks I wrote, tended to match or ever-so-slightly beat FreePascal

The key word here is small. But when Python software grows big, not sure it will happen.

 

 

Edited April 13, 2019 by AlekXL
typos

[AlekXL 8]

3 hours ago, Joseph MItzen said:

As I always say, "Make money with Delphi, not from Delphi". Every library that's sold only raises the barrier to entry for a potential new user. If I was being mean, I could say that Python or javascript developers don't need to sell libraries because they can find jobs.   

better be mean, than dishonest, that's my way to go.

But you are right here, just saying 'A', say 'B', too

Say, some developer Alek wants to fix those issues in FPC -- Delphi incompatibility, and slow math, on x86-64 Win/Lin. This is major effort, which requires months of full-time development, I presume.

Finally he gets jobs done, and what to do now? He cannot sell FPC, since GPL requirement will force him to open the source just first who asks this, and then sales are over.

He only can use the software in-house, but this is not viable option, unless he's part of big company. However, companies are not interested in pascal, it's not trendy. And we have vicious circle -- to kick some momentum we need some major effort, for a free.. That's the catchs of GPL, and why its actually not so popular as before.

 

[AlekXL 8]

3 hours ago, Joseph MItzen said:

There's a long-running open source project whose software automatically wraps C and C++ code to make it callable from other languages! It's called SWIG, or Simplified Wrapper and Interface Generator

Good to know. Yes, SWIG is a way to go, then, clearly

3 hours ago, Joseph MItzen said:

Marco Cantu decided to deny that a problem even existed, insisting that he does not believe that the .NET ecosystem is larger than the Delphi ecosystem (!!!!).

He's hopeless, isn't he? Delphi still have better ecosystem, when it comes to desktop UI, I believe. Never heard about something like "HTML components" developed in C#, Java, let alone Python

[AlekXL 8]

3 hours ago, Joseph MItzen said:

People don't sell each other software libraries anymore.

People don't sell software that much, anymore.. Paid apps market  shrunk dramatically, and web services is a way to go. Not sure whether Delphi would fit on the back-end market.

 

3 hours ago, Joseph MItzen said:

Delphi does almost nothing to make it easier and it's the equivalent of banging on it with a hammer to get C++ code into Delphi. Here's an example of a newer one for Python: PyBind11

How it works? Like IDispatch? Have no idea how to bind C++ object, let alone lambda, to python, or even Delphi code. There should be a catch somewhere.

[Edwin Yip 154]

On 4/12/2019 at 10:05 PM, Ugochukwu Mmaduekwe said:

Yes It is my project and no it doesn't implement Edwards curves.

I'm no expert in the crypto field, how's the SynECC unit?

https://github.com/synopse/mORMot/blob/master/SynEcc.pas