I read a quip from someone today that suggested "modern GPUs" don't need more than 8x PCIe 3.0 lanes (8 GiB)... which amused me since I just discovered my CPU+Motherboard combo (a Ryzen 3400G & B450) was capping my GPU at exactly that.
Speaking of GPUs, I managed to pick up an RTX A4000 with 16 GB of VRAM for a reasonable price (it was a former mining card). It currently trains 4x faster than the GTX 1080, but if "Tensor Cores" specs are to be believed, it should support even more. 🤔
When I dug into it, 8x on the primary 16x slot is a "feature" of the Ryzen 3400G (and older Ryzen CPUs). Most other 3000 series CPUs don't have this limitation. 😑
As for the B450 chipset, some rare motherboards do actually support 16x PCIe 4.0 (it's a CPU feature not chipset), but only with capable CPUs and only in beta firmwares that AMD requested partners not ship with. 😑
Replacing the CPU will double my available bandwidth. Replacing both will quadruple it. I hope doubling is enough. 😅
A frustrating detail I also learned was that until recently, "consumer CPUs" were limited to 20 PCIe lanes per CPU. 4 lanes reserved for use by the motherboard chipset (USB+SATA+M.2), and the remaining 16 lanes were either dedicated to the main 16x slot, or shared across two 16x slots, running at 8x each.
Ryzen 3000+ chips (besides mine) have 24 lanes, and the Ryzen 7000+ chips have 28 lanes. In both cases, 4 lanes are reserved for the chipset, and any remaining sets of 4 lanes go to M.2 slots.
@dpwiz Having a "dedicated box" is the plan. PyTorch has a "job server" feature, so in theory any machine on my network could send jobs to it.
Right now I'm just SSH'ing in from my laptop.