Mating in mammals has a basic asymmetry – females must invest more in each child than males.
Such strong pair-bonds were held together not only by threats of social punishment, but also by strong feelings of attachment. And because they are asymmetric, their betrayal is also asymmetric.
Women betray bonds more by temporarily having fertile sex with other men, while men betray bonds more by directing resources more permanently to other women.
So when farmer husbands and wives watch for signs of betrayal, they watch for different things.
Husbands watch wives more for signs of a temporary inclination toward short-term mating with other men, while wives watch husbands more for signs of an inclination to shift toward a long-term resource-giving bond with other women.
(Of course they both watch for both sorts of inclinations; the issue is emphasis.) This asymmetric watching for signs of betrayal produces asymmetric pressures on appearances.
While a man can be more straight-forward and honest with himself and others about his inclinations toward short-term sex, he should be more careful with the signs he shows about his inclinations toward long term attachments with women.
Similarly, while a woman can be more straight-forward and honest with herself and others about her inclinations toward long-term attachments with men, she should be more careful with the signs she shows about her inclinations toward short term sex with men.
For both men and women, carelessly strong signs of an inclination toward betrayal could needlessly break their marriage.
Of course it may sometimes be in one’s interest to show weak signs of such an inclination, as a threat to induce better terms of trade in the relation.
But such brinksmanship should be done very carefully.
Men and women may have evolved, either genetically or culturally, to adapt to these pressures on their appearances.