Working with supercomputers is a pain in the ass. It's a miracle these things even boot up with so many devices to keep track of, but using software gets far more weird. Half the operations work on one version and another half works on the other, but only if the number of used processors is even... Maybe it's ompi shenanigans, who knows. Weird.
@academicalnerd Oh no, what's the deal? I love working with mine, but maybe our IT team does a better job managing it or uses better software?
If you would like any tips or think I can be of help, just let me know and maybe I can make some suggestions 😄
Mine just randomly crashes like once every ten minutes. SSH session freezes and that's it. And nwchem doesn't give any readable error output if the input file is weird or incorrect, just kinda hangs. Then again, quantum chemistry packages are written mostly by scientists, not by programmers. Hence the weirdness.
@academicalnerd To fix the first issues, you can try the [following](https://serverfault.com/questions/575112/why-do-my-ssh-sessions-freeze-after-some-time) on your local machine. (BTW, if you're not using TRAMP in Emacs to ssh, you're doing it wrong IMO. It's absolutely a killer feature for ssh to improve server interactivity and such)
As for nwchem, I checked out the user manual. Unfortunately, it has nothing on debugging simulations (typical of molecular dynamics and QM software). However, from my own work on a prior thesis, I can make 2 suggestions:
1. Write a translator from a set of functions to the input file format (add some basic error checking if you want, but I had to do this crap for Schrodinger Suite's software: function in R or python with desired inputs that never makes an error when writing the input files for multiple combinations of experiments)
2. Try running the code through gdb with fortran specific options to see if you can catch strange behavior and terminate it without letting things hang. [Here's](https://undo.io/resources/debugging-fortran-code-gdb/) a possible option to get you started if you're unfamiliar, though this may be trickier.
I hope that helps! Supercomputer access should always be fun, I hope this pushes it in that direction for you 😎 😂
