Yeah this brings up one of the axes I like to grind that #Mastodon actively spreads content throughout the system even to people the author does not intend, relying on good faith to not display it. Users really need to know that this system does not do much to protect privacy securely.
Anyway, I would suggest that you go down the road of encryption. You can broadcast and encrypted message all throughout distributed systems even though only the intended readers will be able to decrypt it, with a good UI handling all of the technicalities of course.
That is, as long as you are okay with it being observable that the speaker has spoken.
I just really wish that sort of functionality had been built into #fediverse