So, first of all, thanks for the input, @Pat, @cstanhope, @Segebodo, @timjan, @alinanorakari !
It turns out it was sort of digital but mostly clock related. For some reason, the GCLK Generator 1 was set to 120 MHz (div 1) while in other examples it was set to a lower value of 12 MHz, i.e. higher div (10 for example). Well: now the sawtooth is smooth as a baby's butt.
The thing is: I only found this by going through a dozen graphical representations of the configurations - which were buggy. Not sure why it was GCLK1, by the way. The datasheet says GCLK_DAC must be lower than 12 MHz casually under 47.6.3 Operating conditions ^^ (I set it to 20 MHz (div 6) and it still worked though).
So, at least my hunch was right: This was indeed not a nonmonotonous DAC (because, wow, that would've been awful!).
My second hunch was right too: There's way more experience to gather.
**Lessons learned:**
1. Always make sure your clock settings are okay.
1. Get used to looking for crucial info on speed and timing under sub-paragraph 6.3+ 😵
1. If you can, chose a product with a clean, functional and stable IDE+toolchain (or use your own).
@cweickhmann I'm glad you found it! That must be a relief. And thanks for reporting back. I've been wondering if you'd been able to track down the problem.
@cweickhmann thank you for this short post-mortem. I hope to one day have the tools and skills to debug my own circuits in a similar way. Cheap oscilloscope and logic analyzer have been on my buy list for some time, and soon I'll have a dedicated space where I can leave my hobby projects and the associated tools and devices
@Pat @cstanhope @Segebodo @timjan @alinanorakari
Addendum:
Manufacturers make you jump through hoops ... by faulty silkscreen:
* PB4 marked as PA4
* PB5 (ADC) marked as PA5 (DAC)
Coz' it's not a biggie to mix up input and output </irony>