Wikipedia Gender Bias
Note: This is a modified version of an essay that I had published on a previous blog.
View: Wikipedia gender bias is a small problem; Importance: Low; Confidence: Low
I remember all the articles, some from 10-years ago, complaining and complaining and complaining and complaining about gender bias on Wikipedia. I found this very aggravating. I thought “What?!? Anybody can edit Wikipedia!”
I have felt my mind returning to the Wikipedia issue, seeing it as a quintessential example of excesses in the social justice movement. “You’re just like the people complaining about Wikipedia. You’re just looking to be mad about nonsense.” I noticed that despite the recurrence of this thought pattern, I had never actually thought deeply about whether concerns over Wikipedia gender bias were justified. So I began browsing relevant coverage.
Most surveys find less than 20% of Wikipedia editors are female. There is no shortage of explanations. Common ones include misogyny, personality differences, androcentrism, and unfair rules. The biggest personal lesson I drew from reading the popular media coverage is that, contrary to my initial reaction, most people aren’t that concerned about women missing out on the hedonistic joy of Wikipedia editing. They’re more worried that the lack of gender diversity in Wikipedia’s production makes for a worse encyclopedia. The thought is that the content of Wikipedia will shape the future, so women need to have a voice.
There are some obvious ways in which low female authorship will degrade an encyclopedia’s quality. Wikipedia should serve the informational needs of it’s readership, which is not male-dominated. Too many male editors might lead to too much coverage of male-friendly topics. Men are not going to be as attuned to sexist framing, so an absent female perspective is bad for neutrality.
Sure. Sure. Sure. It’s not hard to persuade me that these problems exist on some level, but are they big problems or small problems? The news coverage says big. Yet, their evidence is mostly anecdotes and stylized examples, rather than a systematic assessment of Wikipedia’s content. Has anyone done a scan through Wikipedia to find evidence of widespread bias?
There is some research sifting Wikipedia’s content for gender bias, focusing specifically on biographies. The authors compare biographical coverage to other sources, such as notable name databases and Google Search Trends. The question is whether notable people missing from Wikipedia are disproportionately female. They also investigate linking patterns between pages to see if female entries tend to be less central or excessively yoked to those of notable men. Lastly, they find gender imbalances in the frequencies of certain words, and then use linguistic engines to group those words into categories (e.g. cognition-related) to see if the findings align with sexism hypotheses.
The clearest finding is that Wikipedia has way more biographies of men than women — about five male entries for every one female. Intuitively, the skew varies by genre. For example, there is a greater gender disparity among military figures than there is among royalty. The research is a bit conflicted on whether notable women are more likely to be missing. In Wagner 2015, the authors found “a slight over-representation of women.” Contrary to feminist expectations, this means that less notable women more easily find their way onto Wikipedia. However, later research by those same authors found the opposite, with Wagner 2016 concluding that “women in Wikipedia are more notable than men.” Reagle 2011 also found worse coverage for women. Even if women effectively require a higher bar for inclusion, this mostly affects the fringes. In absolute terms, Wikipedia has over 200,000 female entries — so anyone missing is going to be pretty obscure.
The metadata, network, and linguistic analyses find scattered examples of apparent bias. Many of the examples appear small and cherrypicked. For example, here is one of the “major findings” in Graells-Garrido 2015:
Sex-related content is more frequent in women biographies than men’s, while cognition-related content is more highlighted in men biographies than women’s.
However, the sex finding only shows up in their “Word Frequency” analysis — not their “Word Burstiness” analysis. Conversely, their cognition effect is strongest in their “Word Burstiness” analysis. Across both tables, they include 40 comparisons, only 5 of which are significant. Yawn.
A headline finding in Wagner 2016 is that “abstract terms tend to be used to describe positive aspects in the biographies of men and negative aspects in the biographies of women.” Of everything I read, this sounds as if it is getting the closest to the sort of subtle biases people worry about:
The Linguistic Intergroup Bias (LIB) theory suggests that for members of our in-group, we tend to describe positive actions and attributes using more abstract language, and their undesirable behaviors and attributes more concretely. In other words, we generalize their success but not their failures. Note that verbs are usually used to make more concrete statements (e.g., “he failed in this play”), while adjectives are often used in abstract statement (e.g. “he is a bad actor”). Conversely, when an out-group individual does or is something desirable, we tend to describe them with more concrete language (we do not generalize their success), whereas their undesirable attributes are encoded more abstractly (we generalize them).
Wagner 2016 scanned Wikipedia for this phenomenon, and report this table:
Abstract positive language was 10% more common in male Wikipedia entries. Interpretation is tricky here. The authors are not actually claiming that positive language is more common in male entries. They’re showing that relative to positive words in female entries, positive words in male entries are more likely to be abstract. By abstract, they mean more likely to be adjectives, relative to be other forms of speech. Adjective ratios proxy abstraction, because adjectives are the most abstract “syntactic class of terms” according to according to the “Linguistic Category Model.” I’m tempted to cue the Twilight Zone music, but it makes sense if you squint. Men and women amaze, but only men are amazing. I see, I see.
The research I have described is based on analyses of hundreds of thousands of articles. This has obvious advantages, but a disadvantage is that it is not a good proxy for how anyone actually engages with Wikipedia. On average, Wikipedia generates a little more than 1,000 monthly page views per article. However, Piccardi 2020 estimated that fewer than 20% of articles get more than 100 monthly views. Many of the ignored articles have not been significantly edited, and may have higher risk of gender bias.
What kind of gender bias would you see if you limited your analysis to only the most important articles? Wikipedia has a “vital article” concept that lists entries that are considered most central to the Wikipedia project. They are supposed to be more heavily maintained and routinely “reviewed,” with the goal for them to all someday be elevated to “Featured Article” status. For example, the figure below shows the sixteen selected “vital” philosophers and social scientists, which includes just one woman. Personally, I would have put a few other social thinkers ahead of Mary Wollstonecraft (see snubs), but I guess that’s why nobody reads my encyclopedia.
This full list includes 116 male biographies and 11 female biographies. The eleven women are Hatshepsut, Joan of Arc, Elizabeth I, Catherine the Great, Mary Wollstonecraft, Sappho, Murasaki Shikibu, Jane Austen, Marie Curie, Emmy Noether, and Frida Kahlo. It’s such short list, you could actually read them all. I started with Hatshepsut.
Hatshepsut was a pharaoh of ancient Egypt. Initially, she was only a regent, tasked with running the country until a young male heir came of age. She came to power as a teenager, but was a natural ruler — reanimating forgotten trade routes, building great temples, and strengthening her family’s dynasty. Eventually, she asserted herself to be pharaoh by divine right. Rather than assassinate the would-be male heir, she groomed him to rule beside her. Together, they presided over a period of peace and wealth, until her death at around age 50.
After her co-king, Thutmose III, took sole control of Egypt, he decided to redecorate. He removed Hatshepsut’s name from the written records. He destroyed her statues and had them buried. He chiseled her image off the stone walls of the monuments she had constructed. As a result, earlier Egyptian scholars assumed Hatshepsut had been only a regent. Only later, by piecing together evidence, did they recognize Hatshepsut’s monarchic rise.
More than any of the media coverage or research, reading Hatshepsut’s story made me believe it was important to have more female Wikipedia editors. Partly, it was the content of Hatshepsut’s story. Thutmose III’s motivations are mysterious, but I believe his attempted erasure of Hatshepsut was connected to her being female. Yet, I was even more moved by this passage:
Hatshepsut had the expedition commemorated in relief at Deir el-Bahari, which is also famous for its realistic depiction of the Queen of the Land of Punt, Queen Ati. The Puntite Queen is portrayed as relatively tall and her physique was generously proportioned, with large breasts and rolls of fat on her body. Due to the fat deposits on her buttocks, it has sometimes been argued that she may have had steatopygia. However, according to the pathologist Marc Armand Ruffer, the main characteristic of a steatopygous woman is a disproportion in size between the buttocks and thighs, which was not the case with Ati. She instead appears to have been generally obese, a condition that was exaggerated by excessive lordosis or curvature of the lower spine.
I did not cherry-pick this entry. It was the first female entry on a list labeled by Wikipedia as “Wikipedia’s most important articles.” Why does the “Major Accomplishments” section of an article about an Egyptian pharaoh include the phases “large breasts”, “rolls of fat on her body”, and “fat deposits on her buttocks?” These terms are not evening describing Hatshepsut. Rather, it’s about the Queen of the Land of Punt who makes no other appearance in the the article. It is as if the writer could not resist an opportunity to call out a woman for being fat. Reading the talk page, other users have noted this strange content on Hatshepsut’s article. The relevant discussion occurred in 2008 — thirteen years ago. Since then, the page has been edited more than 3,000 times. Yet, as of February 2021, the paragraph remains as quoted. Update: The quoted passage on the Puntite Queen was deleted sometime in 2021. An archived version of the article that includes the passage is available here.
I initially felt resistant to the “gender bias on Wikipedia is a big deal” hypothesis. Part of this was driven by me misunderstanding exactly what people were worried about. In addition, I assumed that there would not be hard evidence of any significant harms from this form of bias. Studying this issue, I came to better appreciate these bias concerns and understand the relevant research. Yet, I never felt myself drift from my initial position of skepticism.
It was only when I read about Hatshepsut, that I could feel my mind change. My bias started shifting in favor of Wikipedia gender bias mattering. This is unfortunate, because it’s an obvious over-reaction. You just can’t actually learn that much about an entire encyclopedia from a single paragraph of a single article. My mind changed against my better judgment.
My point in writing this essay is not to persuade you one way or the other about gender bias on Wikipedia. On that, who knows? Rather, my point is to try to reflect on the types of information that drives our biases. We use high-minded argumentation and empirical research to try to persuade others. But often, our own beliefs are heavily-influenced by trivial happenstance. The person who happened to be my boss. The thing that happened to happen to my friend. In my case, the thing I happened to read on Wikipedia. Or all the things that never happened at all.
My point is to offer one more reason to be humble. To remember that, despite my aspirations, I am not a Bayesian calculator that sucks-in data and spits-out well-reasoned beliefs. I am just a stupid person who believes stupid things for stupid reasons.