Question to other server admins who are using ElasticSearch!
How do you keep up with the growth of the ES database? Run extra servers, clean?
🇳🇱
@snder Bigger and better servers. Its costing us several hundred a month to maintain that sort of capability.
Oh wow.. That's a little over budget for me haha! quey is now running on a €20 server plus Wasabi costs..
I've had it running for a while but it eats up diskspace very fast.. :(
@snder Last month cost me about $600 and next month will come down to closer to $200 after i did some cost saving. Luckily we have over 10K in our funding coffers so we are good for a while.
Holy shit man! Are you talking about the qoto instance now or any porjects? :o
@snder Thats Qoto, but keep in mind qoto is intending to be bigger than just a mastodon instance. The intention is to be a full ecosystem for engineering (software and hardware) solutions, specifically the collaboration side.
I see, you're hosting more then only a mastodon instance! Also PeerTube and some other fun project I see ^^
May I ask how did you raise that much funds for qoto? :o
@snder peertube and similar were hosted but currently downa s we are revamping the whole ecosystem. But yea, thats just a tip of the iceburg for what is planned though.
The funds came from both myself and other investors who funded the project we are going to be chatting about. Qoto is one small non-profit component in the bigger picture there.
@inditoot
The cost is almost entierly due to a massive federation footprint on #QOTO. We keep full federation history, index federated toots, and federate with the vast majority of the fediverse.
As a result the database for Masto alone is many terrabytes large and growing rather quickly. Takes a lot of processing power to index the entire fediverse.
@inditoot @snder The server itself is 120$ its the large server, dedicated option through mastodon.host actually (he helps us on day-to-day maintenance, we modify the source ourself and do the testing ourselves).
Then there is about an extra $60 in backup and storage costs. We have multiple-redundant backup.
@snder
Yup We have a whole organizational structure of things going up around this. Cleverthis.com will be the parent company that mostly donates its profits into QOTO. Qoto will be the non-profit that will host all the various collaboration and hosting services for everyone. One component in that will be mastodon. But I want to make the other services (for example gitlab) unified under the same login. So it will appear as though it is one service.
I also have several other companies that will fall under cleverthis.com but that is unrelated to this conversation.
@snder
Well that depends on just what parts you want to get involved in. You have experience with the fediverse so helping out the Qoto side would be great, but its non-profit so probably not the most benefit to you if you arent also a part of the other projects (which i didnt mention in this thread). So really just depends on what you want to help with and how high of a priority it is for you to come out of it making some money.
In my mind the other projects under cleverthis.com are all enhanced and improved by what Qoto will offer. In fact I plan to make Qoto the place where the projects are hosted (as they are all mostly open-source).
So ideally youd be involved int he whole cleverthis.com universe, including qoto, that will be the best way to ensure youll benefit when the commercial part of the venture starts paying off. Plus with Qoto being non-profit at least we can feel like we are giving back at the same time.
Thats my thinking anyway, ultimately where you get involved and to what extent will be largely up to you and what you want to get out of all this.
@snder @inditoot Yea there is a LOT going on on my end. Enough to overwhelm most people. But my gameplan was to just introduce you to everything I have going on, and the people, then seeing what gets you excited and going from there. I'm flexible. I want people to help me doing what they love because then I know they will be worth the time!
I will say the part of the project that already has initial level of investment is the graphactor project which falls under cleverthis.com and will be hosted on Qoto once its up.
That’s a very clever and nice strategy you’ve got going there! I think you’ll get the best out of people and their work if they have fun doing it and with passion!
I’m very excited to see what you have going there and I can’t wait to be a part of it💪🏻😎 Most of the times I’m working until late on such projects for fun and I wish I could make it more then that :-)
Same here! Let’s meet and see what the future has in store for us and the #fediverse!
@freemo
Ah cool! That’s what I was looking for 🙂 I love it, very nice organized and structured. And under what company will we do our work? Cleverthis or QOTO?
@inditoot