It is possible to have a dependency manager that does not have a single point of failure.
There are probably others, and this is not exactly package manager as such, rather a build tool, but Gradle has a good dependency manager. https://docs.gradle.org/current/userguide/declaring_repositories.html
Solution to a server being a point of failure is having multiple servers, and extension of that is that you allow user customizable list of servers (multiple ones) That way, if one server goes down other's would be available and it can be easy to extend the list with new servers if required. Once you have that, you can easily have even your own private server for distributing your own built packages within the company.