"Somehow we have an unconscious 'victim hierarchy' in our heads that make us prone to believe anything [...]"
Perhaps this is true for certain idealists and progressives. But I think we (standard issue males) have a particular weak spot for women, more so than for other marginalized groups.
Which would make sense in a evolutionary psychology perspective - protecting your tribe's most limited reproductive resource (i.e. female fertility) is important to survival, protecting people who are totally foreign to you is not.