There's a lot of discussion of the gender binary going on in this thread, a lot of which is really interesting. But while binary transgender people have been mentioned, I've noticed that no one has mentioned genderqueer people yet. Being genderqueer/non-binary-gendered myself, I thought I would mention that people like me exist, and that we can perhaps be taken as more evidence towards the gender binary being more of a social construct than something hardwired into human biology.
First of all, a primer for anyone who's unfamiliar with the term "genderqueer": http://en.wikipedia.org/wiki/Genderqueer
And a couple more for anyone who's unclear on the idea of gender identity versus gender expression: https://www.genderspectrum.org/understanding-gender (Scroll down to the "Gender Terminology" section for the definitions.)
For me, being genderqueer means that I don't identify as either male or female. Because of that, I definitely think of my gender as separate from my biological sex -- those two things don't match up for me, and never have. That has led me to experience to what's generally called gender dysphoria these days. Basically, it's a feeling that the sex of my physical body doesn't fit with how I feel as a person on the inside. This is an issue that affects the majority of trans* spectrum people, including many genderqueer people.
I'll use some anecdotes to try to illustrate what I'm talking about, since I know this is a strange concept for most people who've never questioned their own gender. When I was a kid, I didn't feel like either a little girl or a little boy. I didn't prefer to play with either girls or boys and didn't really understand why I was supposed to dislike and stop playing with the opposite sex when I reached the age where that became a thing. I played with a pretty wide variety of toys, but tended to avoid anything that felt to me very masculine or feminine -- e.g. playing team sports and playing with dolls had equally low appeal for me. For as long as I can remember, I dreamed of being able to shapeshift so that I wouldn't have to always be the sex I was born as -- so that I wouldn't always been seen as being a girl or a boy 100% of the time. I wanted to be able to switch back and forth, or, best of all, not have to be either.
When I hit puberty and my body started to develop secondary sex characteristics, I absolutely hated it. I hated that I was now much more obviously one sex and not the other, and began to really loathe my body for the way it was changing. (I'm still working through a lot of those issues, but things are slowly getting better now that I at least have a word to describe my gender identity.) But the idea of switching to the opposite sex and being treated as the opposite gender has never seemed like it would be a real escape for me. It doesn't matter whether people are reading and treating me as male for female: in either case, they're probably going to make a lot of assumptions about how a man or a woman "should" behave, unconsciously ascribe those assumptions to me, and possibly treat me negatively for the ways in which I don't live up to them. (This is what's often called "behaviour policing" -- e.g., "You can't wear makeup and like stuffed animals! You're a man!" and "You can't have a buzz cut and work as a mechanic! You're a woman!" Though it's often not that overt. A real-life example I encountered recently is an older transgender woman I was talking to recently, who firmly believes that any transgender woman who goes out without full makeup and her hair done perfectly is "making [trans* women] look bad". To me, all of those feel like specious arguments without any basis in anything except (Western, in my case) cultural constructs. Why do we see these things as inherently masculine and feminine in the first place? And why do so many people seem invested in making sure that people don't express a gender that is considered different from their biological sex? I haven't yet been able to find a convincing reason for why these unspoken rules exist, except, essentially tradition (aka, "but it's always been that way!") And I'm always reluctant to accept tradition as a valid reason for things remaining as they currently are.)
About ten years ago, I finally came across the term "genderqueer" online, and it was an epiphany for me. "So there are other people out there who don't identify as either male or female? It's not just that I'm a freak, or terrible at being the gender I'm "supposed" to be? And I don't have to decide to be the opposite gender just because I'm not comfortable with my current one? Awesome!" It's been a long road, but these days I'm finally out as genderqueer to my friends and family (some of whom are even being wonderful enough to start calling me "they" rather than "he" or "she"), and I'm working on getting my gender expression more in line with my identity. (It's a discovery process, but it's really empowering to finally feel like I can ignore the people who try to tell me that the way I look makes me a bad example of my binary gender. Because I don't have a binary gender, so hah, your criticism means nothing to me anymore!) Probably needless to say at this point, I feel strongly that the gender binary is far more a social construct that something hardwired into the human brain. Otherwise, it seems unlikely that people like me would even exist -- and we definitely do.
So there's my two cents from outside the binary. I hope it's a helpful contribution to the discussion. If anyone has questions, please feel free to ask. I'm generally not easy to offend.