Not the case at all. Remember, biology is way more complicated than you think. As a starter, there is chromosomal and hormonal sex. Chromosomal sex (X and Y chromosomes) generally determines hormonal sex. But… the gene that determines hormonal sex (SRY) can sometimes get knocked off the Y and onto the X. Or off the Y and onto nothing. Or a bunch of other things. And hormonal sex can get disrupted in a bunch of ways, so that you end up with people who are intersex or seem to be cis female until puberty when they grow a penis. Or who appear to be cis female until they have problems conceiving and find out they are XY but don’t have testosterone receptors. Or any of a thousand other things that make sex determination complicated.

Biology is statistical not deterministic and with 7 billion people there are going to be a lot of edge cases.