3… 2… 1… All systems nominal!

The maintenance issue that happened a few days ago has been successfully resolved yesterday. We were monitoring all systems during the last 12 hours and consider the problem fixed.

What happened? There was an overlap of several factors, both technical and managerial:

10th: We started adding a new server and began migrating some bots from one of our existing ones which was 100% full. The transfer had been completed but the old server did not receive the shutdown command so both servers launched at the same time, logging in bots simultaneously.

This attracted the attention of Linden Lab who quickly blocked the affected servers from accessing Second Life. The bots accounts were not affected, only the servers’ addresses were blocked. We immediately contacted LL and received a list of questions to justify the situation.  A reply has been sent to LL.

11th: We got a reply from LL requesting us to implement several adjustments to our system. LL has been immediately notified that we’ve started adjusting our system. These adjustments were done within a few hours.

Since the server block was not lifted yet, we distributed as much bots as possible to our other servers. Unfortunately we were going to install a new server, so there were very few empty bot slots on the other machines.

12th: LL replied and lifted the existing blocks. All servers are up and running now. There are still several adjustments required by LL for a better customer experience, we are working to implement them.

13th: We added a compensation subscription week to all bots that were offline during the outage. Thank you for understanding, we really appreciate your business!