GetIt alternatives

[David Champion 48]

20 hours ago, Lars Fosdal said:

Practices can only improve

An alternate package manager needs to be controlled and run by an organisation outside of Embarcadero which would give the community a little bit more leverage.

[corneliusdavid 220]

2 hours ago, David Champion said:

An alternate package manager needs to be controlled and run by an organisation outside of Embarcadero which would give the community a little bit more leverage.

I agree. Perhaps this will give a popularity/support boost to one of these:

• dpm
• duget

For me, any open source package that I had previously installed via GetIt, I'm now pulling from GitHub. I suppose I should've done that from the beginning as it would always provide the latest version but this has certainly solidified that decision.

[Vandrovnik 215]

2 minutes ago, corneliusdavid said:

For me, any open source package that I had previously installed via GetIt, I'm now pulling from GitHub. I suppose I should've done that from the beginning as it would always provide the latest version but this has certainly solidified that decision.

Same for me - in previous versions (do not know about Delphi 12 - not using it yet), it usually took several weeks to months before they prepared the packages for Getit, that is another reason why not to use Getit and install manually.

[David Heffernan 2353]

1 hour ago, corneliusdavid said:

For me, any open source package that I had previously installed via GetIt, I'm now pulling from GitHub.

I've never understood why anyone would use getit over the main source repo for open source projects

[corneliusdavid 220]

18 minutes ago, David Heffernan said:

I've never understood why anyone would use getit over the main source repo for open source projects

Convenience, laziness, curiosity. I've found myself browsing the GetIt packages (when it worked) and coming across something that looks interesting but I had not seen before. To try it out, it's really easy to just click Install and start using it to see if it's something I want to keep around. After it's installed, it might get "stuck" there out of laziness or because I only used it once or twice. To switch to the original repo means uninstalling the GetIt package, cloning the repo, learning how it's installed or how to access it... not difficult, just steps to take when there's time and it's going to become a steady part of my library.

[Kas Ob. 124]

7 minutes ago, corneliusdavid said:

Convenience, laziness, curiosity.

Can't agree more on these, yet ...

 

GetIt could simply be pulling a well structured and well thought JSON file from specific GitHub, this is Embarcadero managed GitHub repository, it will hold a list of sources from GitHub or any where else, with a mark "Verified or Emb.Ready" or "Not Verified ! use on your own"

 

So in GetIt you will have the Official sources and non-official ones, also it could list stuff like development or beta or debug.... or many many things, simplifying the whole process, anyone can upload or pull a request to a repository and GitIt Maintainer will list them under categories ... many things can be suggested here, the point is user of GitIt will have way more richer experience, resources, EULA ..., with minimal interaction from Emabrcadero staff or timing, also such GitIt can have its own or user supplied repository like the official one, again, here one can put his own code.

 

Is that much or impossible to implement ?, i guess current GetIt took way more time to evolve to be stable (i never used it), yet to fail like that today, after all why it is not opensource itself ? any particular reason ?!

[Vincent Parrett 763]

9 hours ago, David Champion said:

An alternate package manager needs to be controlled and run by an organisation outside of Embarcadero which would give the community a little bit more leverage

I wasn't going to bring it up in this thread.. but since you opened the door...

I've been working on one for a while now https://delphi.dev  - but it's a big project for one person and I have a business to run and a family.

 

Docs  - https://docs.delphi.dev/

Source https://github.com/DelphiPackageManager

Installer - https://github.com/DelphiPackageManager/DPM/releases (codesigned by VSoft Technologies Pty Ltd). 

 

It is working well, but I have yet to implement loading design time packages - the reason for that is a lack of spare time. I did redo the IDE user interface a while back and that part is much easier to use than before (modelled on Jetbrain's Rider nuget support) - I use it daily.

 

DPM supports Delphi XE2 - 12.0 - which should make upgrading to newer delphi versions easy - if the packages you use are available for the newer version of delphi, just open your project in that new version (with dpm installed) and dpm will automatically download and install the packages required for the project (and update search paths accordingly).  

 

 

 

Packages can be hosted by a package server, or a directory (ie network file share). 

 

The website is also a package server - that part is working well, however the ui needs a lot more work so that users can create api keys for pubishing packages. Right now the only packages there are ones I forked and published. 

 

The website is an asp.net 8.0 application (because I have a lot of experience with it) with a react front end, using a postgresql db - running on ubuntu server on a vm on one of my servers in Sydney - not ideal for latency for the rest of the world - it is behind cloudfare but api routes are excluded (cloudfare does help as a cdn for the images and javascript). If interest in/usage of the server actually took off I would move it to a cloud host in the US to improve latency (that would require some funding).  The actual package files are on a cdn (bunny.net) and are regionally cached, so once someone near you has downloaded a package it is cached on their edge and its fast to download. Right now that's not costing me much but if usage ramps up it would. I went with bunny.net purely because of pricing - but apart from the stupid name it's actually a pretty good service. 

 

Recently someone (thanks @Geoffrey Smith ) contributed a tool for creating dpm package definition files (json) and we have been working on that together (it's installed by default now).

 

 

Geoff created this tool to ease creating package definitions for some commercial libraires he is using. 

 

I would love to see more interest in the project and welcome contributions.  

 

[corneliusdavid 220]

3 minutes ago, Vincent Parrett said:

I wasn't going to bring it up in this thread.. but since you opened the door...

This is the perfect time and place to bring it up and get people behind it! I've started reading a little about it and it looks well thought out.

[dummzeuch 1517]

If I remember correctly there was such a project on GitHub called Delphinus (blog). I never tried it and it doesn't appear to have gotten much traction.

 

Edit: Found it and added the link.

Edited February 2 by dummzeuch

[Kas Ob. 124]

11 hours ago, Vincent Parrett said:

I wasn't going to bring it up in this thread.. but since you opened the door...

Thank you, that is awesome !

[Sherlock 663]

Ah, Delphinus. I remember that. Awesome project! But alas, all free projects collide with some restrictions of other "free" projects (note the different frees here). The following part just made it to much of a hassle for me (I am really lazy):

Quote

The current number of repositories already exceeds the number of calls Delphinus can do to the anonymous Github-Api. You’ll have to provide a OAuth-Token linked to a Github-Account.
https://github.com/Memnarch/Delphinus/wiki/Common-Errors

That ended my brief foray into this great project. But it seems really well thought through and implements many of the ideas posted in this thread. Only downside is the Github monoculture.

[Kas Ob. 124]

@Vincent Parrett That is really fucking neat ! the more i look the more brilliant details a see !

It is a master piece done by a master.

 

 

joking part : (hoping it is funny!)

are you telling me this DPM is written in Delphi by Delphi developer in less than a decade !

 

 

@Sherlock there is another way 😎 https://medium.com/@peternjuguna76/hosting-a-json-file-on-github-pages-a-step-by-step-guide-52105a5a393a

 

[Stefan Glienke 2019]

Regardless of which of the mentioned package managers you will be using - all will be open to supply chain attacks.

[Vincent Parrett 763]

9 minutes ago, Stefan Glienke said:

Regardless of which of the mentioned package managers you will be using - all will be open to supply chain attacks.

It is possible to mitigate these attacks with signed packages and package signature verification.

 

https://github.com/DelphiPackageManager/DPM/issues/19

 

https://github.com/NuGet/Home/wiki/Package-Signatures-Technical-Details

 

Ideally a package would be signed by both the developer's certificate and the package servers's (in the case of a public repository). 

 

Nuget uses code signing certifcates, which is still a problem open source developers - they cost money and are next to impossible to get as individuals. 

 

You could also run your own package server in house and scrutinize each package uploaded before making it available to users.  DPM's server does an av scan using clamav - I would like to use something more robust like virustotal but they have limits that cost $$ to increase to practical levels. 

[Sherlock 663]

Moved this part of the discussion to here. Looked smoother.

[Anders Melander 1815]

 

13 hours ago, Vincent Parrett said:

The website is a single point of failure an asp.net 8.0 application (because I have a lot of experience with it) with a react front end, using a postgresql db - running on ubuntu server on a vm on one of my servers in Sydney

Fixed it for you...

 

Am I wrong?

[Vincent Parrett 763]

4 minutes ago, Anders Melander said:

Fixed it for you...

🙄

5 minutes ago, Anders Melander said:

Am I wrong?

 

I'm actively trying to get something going here that is better than what embarcadero will ever offer, what are you doing (apart from picking holes) 🤷‍♂️ 

 

Unlike embarcadero, I'm not running on old extremely difficult to fix hardware - it's a virtual machine (proxmox) and backed up regularly. The plan is eventually to move it to a cloud service which could provide the redundancy needed - gotta start somewhere.  

 

[Anders Melander 1815]

Just now, Vincent Parrett said:

what are you doing (apart from picking holes) 🤷‍♂️ 

Providing critique. Are you opposed to that?

 

2 minutes ago, Vincent Parrett said:

The plan is eventually to move it to a cloud service which could provide the redundancy needed - gotta start somewhere.  

Okay then. I know this was originally designed as something else but IMO, if this is to be the package manager, in the sense discussed here, it would be smart to base it on a distributed architecture with no SPOF.

[Vincent Parrett 763]

Just now, Anders Melander said:

Providing critique. Are you opposed to that?

Contructive critique is fine, sarcastic comments like "fixed it for you" is not at all helpful and frankly annoying.   

1 minute ago, Anders Melander said:

Okay then. I know this was originally designed as something else but IMO, if this is to be the package manager, in the sense discussed here, it would be smart to base it on a distributed architecture with no SPOF.

I have blogged, posted here and on other forums numerous times, created an RFC repo on github for feedback - but very little input was received and there no proposed distributed designs with any detail (let's use github and all our problems will be solved 🙄). 

 

I have been researching package managers for a long time. None are perfect, and dpm certainly isn't nore will it ever be, but it's a lot better than what we have now - getit is really not of any use (other than for installing delphi).    I looked at the package managers for most of the major language/tool eco systems out there - and took ideas from all of them.  None of the package managers I looked at are distrubuted - can you point to any that are?

Distributed sound cool and all - but how do you ensure

 

a) that the package has not been modified by anyone other than the author (Integrity)
b) the package is actually from the author (Authenticity)
c) that the package actually comes from where it says it does (Provenance).

 

It's difficult enough to do with a central package server, doing so in a distrubuted system would be next to impossible (who is the source of the truth?). 

 

DPM does allow you to configure multiple package sources (as to most other package managers) - so you can easily download the package files (or just copy them from the package cache) to a network share and point dpm at that - in fact in some corporate enviroments this would be the only allowed scenario (I have seen this quite a bit with nuget) to avoid supply chain attacks and license issues.  

 

[Anders Melander 1815]

2 minutes ago, Vincent Parrett said:

Contructive critique is fine, sarcastic comments like "fixed it for you" is not at all helpful and frankly annoying.   

Point taken. I apologize.

 

Although I have great interest in you project I haven't gotten involved at all because I simply don't have spare time or room in my head for yet another project.
Anyway, it's not my area of expertise but I would have thought that existing solutions to distributed chain-of-trust could be used. I don't really have time to think that through to an actual solution though, so I can't claim that what I ask (no SPOF) is doable at all.

[Dalija Prasnikar 1404]

1 hour ago, Anders Melander said:

Anyway, it's not my area of expertise but I would have thought that existing solutions to distributed chain-of-trust could be used. I don't really have time to think that through to an actual solution though, so I can't claim that what I ask (no SPOF) is doable at all.

It is possible to have a dependency manager that does not have a single point of failure. 

 

There are probably others, and this is not exactly package manager as such, rather a build tool, but Gradle has a good dependency manager. https://docs.gradle.org/current/userguide/declaring_repositories.html

 

Solution to a server being a point of failure is having multiple servers, and extension of that is that you allow user customizable list of servers (multiple ones) That way, if one server goes down other's would be available and it can be easy to extend the list with new servers if required. Once you have that, you can easily have even your own private server for distributing your own built packages within the company.

[Vincent Parrett 763]

23 hours ago, Anders Melander said:

Point taken. I apologize.

Accepted, moving on.

23 hours ago, Anders Melander said:

distributed chain-of-trust

Did you mean Web of trust - like OpenPGP? https://en.wikipedia.org/wiki/Web_of_trust - I can't imagine how that would work practically in a world wide scenario - we certainly wouldn't be meeting to present our public keys 🤔

 

In any event, it seems unnecessarily complex when simpler solutions to SPOF exist. 

 

21 hours ago, Dalija Prasnikar said:

Solution to a server being a point of failure is having multiple servers, and extension of that is that you allow user customizable list of servers (multiple ones)

Yes, and DPM is absolutely architected to facilitate that (see the second screenshot above).

21 hours ago, Dalija Prasnikar said:

Once you have that, you can easily have even your own private server for distributing your own built packages within the company.

Yep - the source for the DPM website/server is here https://github.com/DelphiPackageManager/DPMGallery (f-react-ui branch) - and it would not be difficult to build/install onsite (though not documented yet). The rest api surface is relatively small (and still evolving) so it's quite conceivable someone could re-implement the server however they like - for example adding authenticated access for downloading packages. The package storage mechanism is abstracted with multiple implementations (Google, Amazon, Bunny.net and FileSystem) so you can choose where the actual package files live.  I originally used S3 but that was expensive so moved to bunny.net which at the moment is only costing me $1 per month!

Edited February 3 by Vincent Parrett
typo

[Jonah Jeleniewski 16]

DPM looks really interesting, kudos! 

 

I'm surprised no one has mentioned Boss, I'm using it for an open-source Delphi/FPC library and have found the experience to be very pleasant.

[corneliusdavid 220]

On 2/3/2024 at 4:11 AM, Jonah Jeleniewski said:

I'm surprised no one has mentioned Boss,

Interesting! And written in Go.

[Vincent Parrett 763]

On 2/3/2024 at 11:11 PM, Jonah Jeleniewski said:

I'm surprised no one has mentioned Boss,

I have looked at Boss in the past, I found it odd that it's written in Go and not Delphi.  

 

It lacks any means of discovering packages, at least I wasn't able to figure that out?