really impressed by numba (jit compiler for python) performance.

I had a piece of rather trivial code, some manipulations with pandas dataframes that was kinda slow and (the most important) despite all my attempts copied the entire df twice. The df in question was about 6G, so its coping was always receiving a lot of attention from the OOM killer.

I rewrote it tonight in fortran-style (well, a scientist can write fortran in any language), slapped @njit on it, preallocated all columns, passed raw numpy arrays inside the function and the problem is gone.

Sign in to participate in the conversation
Qoto Mastodon

QOTO: Question Others to Teach Ourselves
An inclusive, Academic Freedom, instance
All cultures welcome.
Hate speech and harassment strictly forbidden.