what is so wrong for men to wear womens clothes and women to wear mens clothes ?

[Pete Ross Junior]

17. Interesting question, but it gets fuzzy around the edges ...

There are certain kinds of clothes that are canonically women's clothes, such as dresses.
If a man wears those he is a "cross dresser".

But canonical men's clothes have been socially acceptable on a woman since the 1960s.
For example a woman wearing Levis and a work shirt might be stereotyped in some way, but not a "cross dresser".
And a woman's business suit differs from a man's only in cut.

If it were a high class formal occasion, a woman might get some comment if she showed up in a tuxedo, but nowhere near to the degree a man would in an evening gown.

I agree it's not fair, but "cross dressing" is definitionally pretty much a male thing in the prevailing culture.
YMMV with religious communities.

I think the advancing trend toward gender variety is chipping away at it, but we're far from there.