Here’s the New Palgrave Dictionary of Economics definition of statistical discrimination:
Statistical discrimination is a theory of inequality between demographic groups based on stereotypes that do not arise from prejudice or racial and gender bias. When rational, information-seeking decision makers use aggregate group characteristics, such as group averages, to evaluate individual personal characteristics, individuals belonging to different groups may be treated differently even if they share identical observable characteristics in every other aspect. … group-level statistics, such as group averages, are used as a proxy for the individual variables. (source)
Statistical discrimination, also called rational prejudice (or rational racism), occurs when some stereotypes are considered to be statistically “accurate”. Accurate here in the sense of statistical generalization or average behavior (or characteristics).
Young drivers are on average more likely to be involved in traffic accidents. Hence, auto insurers impose higher auto insurance premiums on young people. This insurance discrimination is called statistical discrimination because it’s based on some level of accuracy regarding the stereotype, and not on plain prejudice of or disgust for young people.
Regarding life insurance premiums, the customer’s likelihood of dying is one of the most relevant variables affecting the insurers profitability. Since gender is highly correlated with life expectancy, it is therefore optimal for the company to adopt a policy setting higher premiums for men, even for those men who may be healthier and less risk prone than the average woman. (source)
Employers often use the average performance of an applicant’s group to determine whether to hire him or her.
You don’t hire a guy with a Mohawk as a receptionist at a law firm – even if he promises to get a hair cut. Why not? Because on average, … guys with Mohawks have trouble with authority. (source)
Similarly, people with facial tattoos are unlikely to be hired as CEO’s, perhaps because they signal an anarchic, counter-cultural, contrarian, nonconformist, bohemian and “wandering” attitude.
The reason people engage in statistical discrimination is the high cost of making case-by-case judgments and the relative accuracy of group averages. Given that it’s not a product of bigotry, irrational prejudice, hatred and dislike, is it correct to call it discrimination? It is in the sense that people judge other people simply on the basis of group membership, not individual merit, as in “regular”, “irrational” discrimination or prejudice. Yet it’s not discrimination because people engaging in statistical discrimination often don’t do so because of their dislike or hatred of the discriminated group but because of economic self-interest and risk avoidance, as is shown by the examples above. The stereotypes are not baseless as most stereotypes are (“Blacks are lazy”, “Jews are money hungry”, “women are only productive at home” etc.).
This discrimination is “statistical” because contrary to normal prejudice it acknowledges that a relatively large number of members of a group do not conform to the stereotype. Just as it isn’t irrational to claim that it’s statistically accurate that republicans in general are pro-life while at the same time knowing that many aren’t. “Irrational” prejudice doesn’t have so much tolerance for exceptions. “People see others as average members of their groups until proven otherwise”. (source)
Is it acceptable?
So we know what it is and why people do it, but is it morally and legally acceptable? I think it depends on the type.
While such discrimination is legal in some cases (e.g., insurance markets), it is illegal and/or controversial in others (e.g., racial profiling). (source)
The difference between acceptable and unacceptable statistical discrimination hinges on several facts, I think:
1. How accurate are the group generalizations? You can forgive profit-seeking insurers for imposing higher auto insurance premiums on young people. However, wage gaps between genders resulting from theories about lower productivity of women (said to result from higher female investment in child rearing and a focus on “caring jobs” as opposed to jobs that tend to pay well) are less acceptable because the generalization is more dubious and the category of “women” too broad. Generalizations may be more accurate when they are not about behavior but about more “technical” and less controversial things such as life expectancy.
2. What is the “weight of history”? Statistical discrimination of blacks or women is by definition less acceptable than statistical discrimination of young people, given the history (maybe that’s a stereotype about discrimination…). Differentiated treatment, even if it isn’t really discrimination, can still cause harm given a certain history. The harm can be feelings of insecurity regarding a possible repetition of the past, feelings of continued attacks on self-esteem etc.
3. What are the consequences of statistical discrimination? Paying a slightly higher insurance premium is a lesser harm than being pulled over by the police a few times a week because of your skin color, even if your skin color is a statistically “accurate” prediction of the probability that you engage in crime in your neighborhood. (More on racial profiling).
4. To what extent is statistical discrimination self-fulfilling? To some extent it clearly is.
People think teen-age males are criminally inclined (and they are), this angers the teen-age males, leading them to commit more crimes. (source).
If everyone assumes that boys are not good at domestic work, the benefit of learning domestic work is zero because no one will hire you as a domestic worker. In some areas, this self-fulfilling process is obviously harmful, and hence statistical discrimination is harmful as well.
It is possible that, because employers are less likely to hire women in jobs that require labour market attachment, then women are more likely to be involved in child-rearing than men, and are less prone to acquire the skills that are necessary to seek and perform well in those jobs, confirming the asymmetric belief that employers hold regarding labour market attachment. (source)
This phenomenon is called the stereotype threat. On the other hand, there can be self-reversing prophesies at work as well. A-typical members of a group, who of course suffer from the prejudice in the same way as typical members, may incite the latter to change their behavior. E.g. responsible young drivers who pay too much insurance because of the typical behavior of their fellow group members, may pressure the latter to change their ways. In this way, statistical discrimination may work to undermine itself. The same result will occur because a non-typical member of a group may get relatively more exposure. A woman who’s relatively good at soccer – compared to other women – but just as good as an average male soccer player, may get more attention than the latter. This excess of attention may undermine the stereotype on female soccer players.
More on human rights and risk is here. More on prejudice is here.