Some of the following factors may contribute to your unhappiness
Our current overemphasis on appearance
Society is probably more fixated on physical appearance at the moment than at any previous times. Great care is taken by all celebrities to ensure they look perfect when they are seen and to suppress all pictures of them looking less than ideal. The proliferation of these unreal images is thought to be linked with eating disorders and with many ordinary people's dissatisfaction with their own appearance.
Our poor understanding of what constitutes attractiveness as opposed to beauty
Defining why we find other people attractive is difficult. In long-term studies of happy couples, the qualities that people find attractive seem to include kindness and thoughtfulness, humour and carefreeness, shared interests and common goals. However these qualities are quite elusive and difficult to define, so there is a natural tendency to overestimate the importance of physical good looks which are more easily quantified and readily visible.
Getting caught in the trap of depression
People who are happy, carefree and spontaneous tend to be more attractive to others than those who are sad and preoccupied. They also look physically more attractive. Therefore a vicious circle can develop in which we become depressed because we decide we are unattractive; then we look less attractive because we are depressed. This is even more likely to be the case if being depressed means we are not taking care of ourselves.
Not being able to get past first impressions
One could argue that strikingly beautiful or handsome people do not have to find ways to meet people or start conversations, as they are more likely to be approached by others. Even if this is true, introductions are of course only the first part of a relationship, and once we meet someone our estimation of them usually changes. However, getting to know people is one of the harder parts of forming relationships and takes courage. If we have convinced ourselves that physical appearance is all that matters and that we are deficient in that area, we are less likely to get over the first hurdle. If it is any consolation, people who are strikingly attractive frequently report the opposite problem - that they feel intensely lonely because they find it hard to get people to take them seriously and to move beyond small talk and casual sex in order to enter into deeper relationships.
Memories and associations
If we have been mistreated, bullied or taunted in the past because of some aspect of our appearance, we can end up continuing to feel dissatisfied with ourselves rather than placing the blame where it really lies - with those who acted in such a cruel way. Similarly, aspects of our appearance can remind us of unhappy times. For example, if we have been through a long period of loneliness, we may well wear a habitually sorrowful or closed expression. If we then label ourselves as unattractive, we may then not give ourselves the chance of replacing the old unhappy experiences with new positive ones. Finally, family features - shape of nose, colour of hair - may bring to mind others in the family with whom we have had difficult relationships.