Well I think I finally got to the bottom of why the matrix server has been slow in the past. Now that i have good stats on resources I can better address it... Should have matrix running much smoother soon.

@trinsec

@freemo @trinsec I do hear the matrix developers say there will be a go implementation to replace the slow and resource demanding python version. Is that still in alpha stage?

(As a jealous JVM lover, I don't trust Python 😂 )

@skyblond

That I have no idea.. But if they think go automatically means faster then it will probably be an abysmal failure.

@trinsec

@freemo @trinsec Based on my limited knowledge of python, the multithreading part is pretty heavy, if I recall correctly, you need a new python process to start a new thread (sounds familiar with JVM :ablobthinking: ). And go is pretty good at multithreading (I mean user-mode threads). If not limited by the IO, I would assume a go implementation will speed up some of the process. Maybe also ease the load on developers, considering go offers some great built-in multithreading structures.

@skyblond

multithreading is ugly, but not exactly heavy... you can do threading without multiprocess but you have to disengage the GIL, which involves a little bit of magic... its an ugly pattern to be sure, but not resource intensive.

@trinsec

Follow

@freemo ok, thanks, that's some updates to my knowledge base :ablobthinking:

Based on my experience on JVM, when I switched from JVM's native thread to Kotlin coroutines (which is based on threads but is able to share threads, so less thread suspension overall), I got a free performance boost. I assume go can achieve the similar thing. If so, I would say there is a free optimization without largely redesigning the algorithm.

Also, I always prefer strong typed languages when co-op with other developers. Python makes me panic when I don't know what the type of variable x :ablobgrimace:

@skyblond JVM? Thats Java, thought we were talking about python?

Sounds like you implemented threading poorly in java and moved to a language where the way you built it was more appropriate to that language... I mean sure, they might be able to write code on Rust better than in Python simply because they are better at writing code for Rust than they are for Python... but thats not the language's fault.

> Also, I always prefer strong typed languages when co-op with other developers. Python makes me panic when I don't know what the type of variable x :ablobgrimace:

Based on the context here you mean staticly typed, not strongly typed

@freemo oh.. my bad. I did a quick google search and found Python is quite different from JVM. On JVM, a thread is always a kernel thread. On Python, it might or might not be (StackOverflow told me this). So when I initially thought about GIL, I thought it would be act like a monitor lock on JVM, which cause a kernel thread to suspend and have a relatively huge penalty.

And after another goolge search, yes, I do mean statically typed and strongly typed, where you need to declare the type of a variable and cannot change it on the fly.

@skyblond Yea in java multithreading with the purpose of leveraging multiple CPUs is pretty straightforward and similar to any other language.

Python really is the odd one out where multithreading is a bit of a cop-out as it doesnt actually run in parallel and across CPUs and thus requires either C-python to bypass the GIL or multi-process handling.. which is quite ugly to do efficiently.

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.