FYI - Several Embarcadero services are currently unavailable

[Vandrovnik 214]

2 hours ago, Ian Barker said:

Servers... plural. That's the problem. If it was a single server that had gone bang, I think it would have been considerably easier. I don't know the full details, but it was a cascade of events and multiple affected servers and hardware.

Several hardware problems at the same moment? My imagination offers me something like this:

(image found on the web, not from Emba!)

[JohnLM 14]

In the Welcome screen, the Learn videos portion is available and comes up on my 6GB Win10 tablet, under Delphi 11 and 12.  I have both those IDE's working fine together. Infact, have them both running right now. But on my Win7 laptop with D11, the Learn videos portion does not come up. Actually, it has not come up in many weeks or longer. I prefer to watch videos on my 15" win7 laptop.  Yes!  I still use Windows 7 !!  I have a few Windows 10 devices. But Win7 is still my main machine.

 

@Vandrovnik  so funny, LoL

 

 

Edited January 22 by JohnLM

[ioan 45]

4 hours ago, Ian Barker said:

Servers... plural. That's the problem. If it was a single server that had gone bang, I think it would have been considerably easier. I don't know the full details, but it was a cascade of events and multiple affected servers and hardware.

I don't buy it, I don't think this is just hardware failure. My conspiracy theory is that it started with hardware failure and then they realized that the backup didn't backup for 20 years.

No matter what hardware failure you have, it doesn't take two weeks to get it back up and running. 

Edited January 22 by ioan

[Attila Kovacs 629]

1 hour ago, ioan said:

 No matter what hardware failure you have, it doesn't take two weeks to get it back up and running. 

It's sufficient if there was an older operating system or a part of the system that won't run on current hardware, and you can't simply restore and go.

[Lars Fosdal 1792]

Due to the supply chain disturbances, we had a 5 month delivery wait for servers and parts for a while, so I can relate to delays.

 

The problems here are:

- Lack of proper communication. "No news. Next update at ..." is better than silence!

- Insufficient contingency planning? 

- Insufficient backup and recovery plans?

- No plan B? There are many capable cloud hosting services that doesn't take forever to spin up a pack of servers.

 

 The cost might be high, sure - but so is the cost of losing the trust of your customers.

[Vincent Parrett 750]

14 minutes ago, Lars Fosdal said:

Due to the supply chain disturbances, we had a 5 month delivery wait for servers and parts for a while, so I can relate to delays.

We had the same issue last year, ordered a server in July, turned up in November. At least in our case, this was an upgrade not an emergency replacement. 

 

17 minutes ago, Lars Fosdal said:

No plan B? There are many capable cloud hosting services that doesn't take forever to spin up a pack of servers.

I have to agree - how hard is it to spin up a tempory vm on a cloud service - I suspect there is more to the story but we will probably never know. 

 

[Attila Kovacs 629]

10 minutes ago, Vincent Parrett said:

I have to agree - how hard is it to spin up a tempory vm on a cloud service - I suspect there is more to the story but we will probably never know. 

And then what? What will be served? The default apache landing page?

 

[Vincent Parrett 750]

5 minutes ago, Attila Kovacs said:

And then what? What will be served? The default apache landing page?

No that would be pointless - restore from backup onto temp server. 

[tgbs 14]

14 hours ago, Ian Barker said:

Well, by that I assume you mean you're getting marketing emails. Yes, they are on a different system. So are the blogs and a few other servers and services - for example QP and my.embarcadero.com download servers are working.

For the email you don't want there is always an unsubscribe link - if you click on that they will remove you from the ones you don't want - in fact it's a legal requirement in most countries that they do.

I cannot register for any webinar clicking on link from email

Unable to connect

An error occurred during a connection to lp.embarcadero.com.

    The site could be temporarily unavailable or too busy. Try again in a few moments.
    If you are unable to load any pages, check your computer’s network connection.
    If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the web.

 

[Cristian Peța 103]

51 minutes ago, tgbs said:

I cannot register for any webinar clicking on link from email

The link below is working from here.

https://lp.embarcadero.com/webinar-registration?utm_source=Eloqua&utm_medium=email&utm_content=Webinar-240119-HowToWebinars

[tgbs 14]

Yes sorry. My home antiads dnsmasq is marking lp.embarcadero.com. It is automatic script...

[Remy Lebeau 1393]

On 1/18/2024 at 10:36 PM, zed said:

https://stats.uptimerobot.com/3yP3quwNW/780058619

The parent list (https://stats.uptimerobot.com/3yP3quwNW) doesn't appear to include any GetIt servers for monitoring.  Unless maybe they are listed under a different name (Downloads, perhaps?)

[Silver Black 23]

[Gord P 14]

Rad Studio Wiki is up and running for me

[ginnix 2]

Just tried Rad Studio Wiki and it is also working for me. Hope they have this all fixed up soon.

[Remy Lebeau 1393]

2 hours ago, Gord P said:

Rad Studio Wiki is up and running for me

It is "up", but it is not entirely stable.  I was just using it a few minutes ago and still got a few errors on it.  But yes, I could access pages.

Edited January 23 by Remy Lebeau

[Rollo62 536]

At least, DocWiki looks OK to me. Praise the Lord, it's back again

[David Heffernan 2345]

It's ironic that the download spikes caused by all the Delphi 12 downloads have caused this

[Anders Melander 1782]

2 hours ago, David Heffernan said:

It's ironic that the download spikes caused by all the Delphi 12 downloads have caused this

Yeah. I could maybe forgive them their general incompetence as it's probably caused by a lack of resources - but that lie is just insulting.

[dummzeuch 1505]

Maybe it wasn't the downloads of Delphi 12 (ISO / Installer) but the downloads caused by Delphi 12 itself (GetIt + other stuff in the Startup screen) on every IDE start? That's a kind of plausible interpretation of the wording.

It might not have the been the sheer data volume either but the number of clients that tried to connect.

 

But unless somebody leaks the actual information (or maybe Embarcadero decides to be frank about it (fat chance)), we will never know. So this is just idle speculation.

[Attila Kovacs 629]

29 minutes ago, dummzeuch said:

That's a kind of plausible interpretation of the wording.

Or it was Download Spike

 

[Rollo62 536]

14 hours ago, David Heffernan said:

It's ironic that the download spikes caused by all the Delphi 12 downloads have caused this

Why, this is a typical problem for young start-ups that have developed brand new and fancy products that have suddenly and unexpectedly gone incredibly viral,
but unfortunately, they have not 100% stable infrastructure and business processes yet.

Their main goal would be to stabilize all processes and infrastructures as quickly as possible so that they do not disrupt and hinder the delivery processes created by the hype surrounding their fancy products.

Edited January 25 by Rollo62

[Angus Robertson 574]

This may be a silly question, but did any Embarcadero.com servers ever come back? Two weeks ago...

 

Trying to install a new version, the online installer complains get-it-olympus.embarcadero.com timeout.

 

Angus

 

[Uwe Raabe 2057]

The blog post recently got an update: https://blogs.embarcadero.com/we-are-experiencing-a-hardware-outage/

[Dalija Prasnikar 1396]

12 minutes ago, Angus Robertson said:

Trying to install a new version, the online installer complains get-it-olympus.embarcadero.com timeout.

https://blogs.embarcadero.com/we-are-experiencing-a-hardware-outage/ GetIt is still off line, but you can download RADStudio 12 ISO. There is a download link in that blog post.