Programming Our Biases
The Moral Machine’s Fatphobia Problem
A self-driving car is about to crash on a busy street. Who lives and who dies? Through its Moral Machine experiment, MIT researchers analyzed 40 million decisions made by 2.3 million participants in 233 countries to answer just that question.
As a designer and strategic foresight nerd I’m interested in the ways in which our biases might be recreated and literally programmed into the technologies of the future.
What I find particularly troubling in examining the results of the Moral Machine study is the way that overweight or “large” men and women rank lower than their fit counterparts.
To be clear, the researchers who led the study are not advocating that policy decisions be made solely on the basis of their experiment. But, the decision of who lives and dies is one that will need to be made somehow.
I was inspired by past studies that analyzed the Moral Machine data and found that countries with more individualistic cultures prioritized sparing the young. I wanted to uncover a similar explanation for why some countries opted to spare the fit.
I set out to explore whether there is a correlation between a country’s proclivity for sparing the fit and its obesity rates, GDP per capita, and even the number of Olympic Medals they’ve won.
I view this work as breaking new ground at the intersection of technology ethics and fatphobia.
Research Questions
Question 1: What is the relationship between the number of Olympic Medals a country has won and its proclivity to spare the fit?
Q1 Hypothesis: Countries with more Olympic medals place a higher value on athletics and will be more likely to spare the fit.
Question 2: What is the relationship between a country’s obesity rate and its proclivity to spare the fit?
Q2 Hypothesis: Countries with a higher % of obese individuals will be more empathetic and less likely to spare the fit.
Question 3: What is the relationship between GDP per Capita of a country and its proclivity to spare the fit?
Q3 Hypothesis: Countries with a higher GDP per Capita have more workers in jobs where fitness is less directly connected with productivity and therefore will be less likely to prioritize sparing the fit.
Data Analysis
I combined the four data sources below to create my final dataset of 94 countries.
- The Moral Machine Data, MIT
- Global Obesity Data, Our World In Data
- GDP Per Capita Data, Our World in Data
- Olympic Medal Counts, World Population Review
I started by cross referencing a list of 134 countries that have won Olympic medals with the 117 countries that had Moral Machine data. I found that a lot of African countries had won medals, but hadn’t gotten enough traction in the Moral Machine survey to be included. After combining these first two datasets I had 99 countries, but had to remove another 3 that didn’t have obesity data and 2 that didn’t have GDP per capita data. This left me with 94 countries.
I then proceeded to rank the 94 countries in each of the 4 variables: olympic medal count, proclivity to spare the fit, prevalence of obesity, and GDP per capita. I found that putting the data into rank format allowed me to visualize it easier and compare apples to apples. My research questions were concerned with relativity and the relationship between variables (e.g. is a country that is higher in X variable also higher in Y variable) so the exact number of things like gold medals won was irrelevant.
As I cleaned the data I encountered issues with country names being different between datasets. For example, the Czech Republic was listed as Czechia in one dataset. I also had to make decisions like whether or not to re-rank the Moral Machine data, because it originated in rank format, but with a group of 117 countries. I decided to re-rank because I felt it was best to have the rank for all four variables be based on the same set of 94 countries.
Research Findings
The following are data visualizations I created to explore the relationship between proclivity to spare the fit and the following variables: 1) Olympic Medals, 2) Prevalence of Obesity, and 3) GDP per Capita.
I used a scatter plot diagram to show overall correlation between the two variables and a range plot to show the difference in rank for each country. I have included a brief analysis after each visualization. I have also made these publicly available and editable through Datawrapper’s River (search “Moral Machine”).
Research Question 1: Spare the Fit / Olympic Medals
Overall I would say there is a low correlation between Olympic medals and proclivity to spare the fit. If there were high correlation we would expect to see the data points in more of a line from bottom left to top right. Looking at the various quadrants of the chart, in the bottom left we see the United Kingdom is high in both medals and proclivity to spare the fit. It could be that the hypothesis around a culture of athleticism leading to a higher likelihood to spare the fit is true there. But overall it would be difficult to make that assertion.
What I like about seeing the data in this format is you can more easily examine it on a country by country basis and also look at the top 20 and bottom 20 for the Olympic medal variable. If we look at the top 20 medal winners only 4 countries rank in the top 20 for both variables and out of those, only 2 (Romania and Bulgaria) have a proclivity to spare the fit that is higher in rank than their Olympic performance. Countries like Germany, China, and Norway are top 10 medal winners, but in the bottom half of the group (over the 47th rank) in terms of sparing the fit.
In the bottom 20 of medal winners we see countries like Guatemala and Paraguay that rank 4th and 5th respectively in terms of sparing the fit. This indicates that even though these are not sports powerhouses they still have a high proclivity to spare the fit. Why that is, I can’t say.
Research Question 2: Spare the Fit / Prevalence of Obesity
Again, there seems to be little correlation between the prevalence of obesity and proclivity to spare the fit. My initial hypothesis was that countries with higher obesity rates would be more empathetic to overweight individuals. It seems that may be true in Kuwait — the country with the highest obesity rate– as their proclivity to spare the fit is towards the lower end of the spectrum (79th out of 94). But, that doesn’t seem to be the case in a country like Qatar, which is 5th in obesity and 6th in proclivity to spare the fit. Mongolia is also an interesting case where obesity rates are relatively low, but proclivity to spare the fit is very high (ranked 1st). Overall, the scatter plot shows us the data is too distributed to make any sweeping conclusions about correlation.
I find it interesting that the top 5 for obesity rates include 4 countries from the Middle East and that Kuwait, Saudi Arabia and Jordan are all towards the bottom of the list when it comes to sparing the fit. Saudi Arabia is 4th highest for obesity rates, but 92 out of 94 countries in proclivity to spare the fit. Qatar seems to be the outlier amongst those Middle Eastern countries.
Looking down the list, there are 5 countries where the rank for both variables is off by 1 place or less. For example, Australia is 14th in obesity rates and 15th in proclivity to spare the fit. There are even two cases (Tunisia and Luxembourg) where the ranks match exactly. In the Olympic medal chart there was only 1 country where this was the case. The trend line in the scatter plot for obesity is also slightly steeper than the Olympic one, but again I think it would be a stretch to say the two variables are correlated.
Research Question 3: Spare the Fit / GDP per Capita
Finally, my hypothesis around GDP per capita was that countries with higher GDP would be less likely to favor sparing the fit. If this were the case we would expect to see the data arranged in a more consistent line from top left to bottom right. Essentially, countries like Nigeria with low GDP per capita should be in the top left quadrant with Guatemala, rather than the top right. The UK and Qatar should also be in the bottom right quadrant near South Korea.
Overall, that doesn’t seem to be the case and again we can’t make any broad claims about correlation between the two variables.
Looking at the top 2 and bottom 2 countries in this chart illustrates how difficult it is to see a connection between these variables. Qatar is #1 in GDP per capita, but 6th in sparing the fit. If you look at them in a vacuum you might think rich countries also value sparing the fit. But hold on, Luxembourg is #2 in GDP per capita and 45th in sparing the fit, which puts it pretty much dead center in terms of sparing the fit or overweight.
At the other end of the spectrum, Afghanistan is last in GDP per capita and 17th in sparing the fit, which could give some credence to the hypothesis that poorer countries value fit individuals who can contribute manual labor to the economy. But then Kenya is 2nd to last in GDP per capita AND 2nd to last in sparing the fit. So, as with medal count and obesity rates I don’t think we can establish a clear correlation.
Conclusion
To bring it back to my initial interest in this area, the fact remains that in their analysis of 2.3M survey responses the Moral Machine found a preference for sparing fit individuals over large individuals. I was unable to uncover a variable that was correlated with that preference and my initial hypotheses were disproven. I wanted to understand what factors might be contributing to that preference in order to inform policy decisions that could lead to a more equitable treatment of individuals regardless of their size.
I think this is an area that warrants further study. Without such research, we run the risk of programming our biases into future technologies. Imagine if the researchers at MIT had included scenarios where respondents had to decide whether to spare a Black or White individual.
Reflection
Throughout this project I checked my work with a colleague to determine whether they were able to follow the story and to see if they interpreted the findings in a similar way. Initially, I was going to only use the data plot visualization format, but I added the scatter plot on the recommendation of my colleague. She felt it was the best way to quickly show whether or not there was correlation between the two variables and I agreed.
The other main piece of feedback was around how I originally worded the hypotheses. My research interest was tied to issues of fatphobia, and that was reflected in the initial hypotheses where I framed it as more Olympic medals leading to a lower likelihood to spare overweight individuals. This caused confusion because the data was labeled as sparing the fit. So in the visualization, more medals and higher likelihood to spare the fit would prove my hypothesis around lower likelihood of sparing overweight individuals. This little bit of mental gymnastics was an unnecessary distraction, so I reframed the hypotheses.
Based on discussions with my colleague it also seemed that the ranked data caused some confusion because a lower rank in Olympic medals means more medals won, but a lower rank in sparing the fit means a higher rank for killing the overweight individual. I tried to address this by adding a note at the bottom of the scatterplot diagrams.
If I were to do this project over I would consider using a smaller sample size. 94 countries made the graphics pretty large and the data a little hard to absorb at a glance.
If you have ideas for how to segment the data further or additional analyses that could be done, I have made the data and charts publicly available in the Datawrapper River. You can find them by searching Moral Machine.
Thank you.
