@freemo @trinsec Based on my limited knowledge of python, the multithreading part is pretty heavy, if I recall correctly, you need a new python process to start a new thread (sounds familiar with JVM ). And go is pretty good at multithreading (I mean user-mode threads). If not limited by the IO, I would assume a go implementation will speed up some of the process. Maybe also ease the load on developers, considering go offers some great built-in multithreading structures.
@freemo ok, thanks, that's some updates to my knowledge base
Based on my experience on JVM, when I switched from JVM's native thread to Kotlin coroutines (which is based on threads but is able to share threads, so less thread suspension overall), I got a free performance boost. I assume go can achieve the similar thing. If so, I would say there is a free optimization without largely redesigning the algorithm.
Also, I always prefer strong typed languages when co-op with other developers. Python makes me panic when I don't know what the type of variable x
@freemo oh.. my bad. I did a quick google search and found Python is quite different from JVM. On JVM, a thread is always a kernel thread. On Python, it might or might not be (StackOverflow told me this). So when I initially thought about GIL, I thought it would be act like a monitor lock on JVM, which cause a kernel thread to suspend and have a relatively huge penalty.
And after another goolge search, yes, I do mean statically typed and strongly typed, where you need to declare the type of a variable and cannot change it on the fly.
@skyblond Yea in java multithreading with the purpose of leveraging multiple CPUs is pretty straightforward and similar to any other language.
Python really is the odd one out where multithreading is a bit of a cop-out as it doesnt actually run in parallel and across CPUs and thus requires either C-python to bypass the GIL or multi-process handling.. which is quite ugly to do efficiently.
@skyblond JVM? Thats Java, thought we were talking about python?
Sounds like you implemented threading poorly in java and moved to a language where the way you built it was more appropriate to that language... I mean sure, they might be able to write code on Rust better than in Python simply because they are better at writing code for Rust than they are for Python... but thats not the language's fault.
> Also, I always prefer strong typed languages when co-op with other developers. Python makes me panic when I don't know what the type of variable x
Based on the context here you mean staticly typed, not strongly typed