What do you trust over a tool that hasn't earned ground truth?

Been reading about AI doing code reviews and it just makes me think of rookies who called in a fire line from the map instead of walking it. When has your gut beaten a fancy system, and when did the system turn out to be right?