Minor Update: So yesterday we had to tweak some resources server side, despite fixing the 503 with more resources things were still a bit sluggish. We boosted a few more resources (expanded the cluster itself) and now the system seems quite responsive as well.

Please let @freemo know if you expiernce sluggish behavior or errors.

Only thing left to fix that is broken is some UI issues with links. Some of that will actually expose features that are technically working like editing in place and translation capabilities. So as long as nothing else is broken that is more pressing we should have that fixed up in short order and then we can start working on the next version again and get out of cleanup mode.

So seems the upgrade went ok, a few small issues we should have fixed up pretty quick. Here are the broad points.

* This was our first big move over to an entirely new architecture meant for quick upgrades with no down time. Ironically this meant a 2 day down time but the advantage is future upgrades should have very minimal downtime now.

* We Did 5 upgrades in one go, lots of new features as a result, mostly all the new stuff from mastodon combined with the features QOTO already had.

* The new setup was having some 503 errors at first, and it was a out of memory issue. We have since significantly boosted our resources. The system is now much faster and responsive.

Some issues that remain, and when we will have it fixed:

* There are a few small UI issues, the link to your own profile points to the wrong link, and the "local timeline" link is missing to bring it up. We should have those back in place very shortly (probably a day or two).

* There were 503 issues periodically but it should now be fixed, please let me know if anyone sees a 503 again.

* Editing posts in place (instead of delete & edit) now works but it is missing from the menu, so we need to add that.

* Translation features should also be in place, but lacks the link.

If anyone notices any other issues message @freemo . In the meantime give us a day or two and we should have these small issues all resolved, thanks.

Just an FYI, on Thursday (2 days) we will be re-attemping our 5-in-1 update. It will be a very major update so expected as much as 3-days downtime as we are finally moving to the new architecture. We **will** be up at the other end with hopefully everything working and a few new features.

Seems our load balancer is a little under powered and its stalling momentarily from time to time. Most of you probably didnt even notice it but some of you may have.

I am going to upgrade that today to the next level. May result in up to 5 minutes down time (the new architecture will have multiple load balancers).

Either some of the fixes I did yesterday wound up addressing the latency problem we had, or it was intermitten. But everything has been operating fine for the last 12 hours or so. I will continue to monitor the situation but for now everything is in the green on

Assuming no more problems trigger we and the team are continuing on building our beta environment which will enable for smooth update role outs int he near future. We have about 5 updates we developed in house we want to apply in the near future.

Show older
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.