USA: One of the most exciting and well-attended events at the International Conference on Machine Learning in July was called “Participatory Approaches to Machine Learning.” This workshop tapped into the community’s aspiration to build more democratic, cooperative, and equitable algorithmic systems by incorporating participatory methods into their design.
Such methods bring those who interact with and are affected by an algorithmic system into the design process—for example, asking nurses and doctors to help develop a sepsis detection tool. This is a much-needed intervention in the field of machine learning, which can be excessively hierarchical and homogenous.
But it is no silver bullet: in fact, “participation-washing” could become the field’s next dangerous fad. That’s what I, along with my coauthors Emanuel Moss, Olaitan Awomolo, and Laura Forlano, argue in our recent paper “Participation is not a design fix for machine learning.” Ignoring patterns of systemic oppression and privilege leads to unaccountable machine-learning systems that are deeply opaque and unfair. These patterns have permeated the field for the last 30 years. Meanwhile, the world has watched the exponential growth of wealth inequality and fossil-fuel-driven climate change.
These problems are rooted in a key dynamic of capitalism: extraction. Participation, too, is often based on the same extractive logic, especially when it comes to machine learning. Participation isn’t free Let’s start with this observation: participation is already a big part of machine learning but in problematic ways. One way is participation as work.
Whether or not their work is acknowledged, many participants play an important role in producing data that are used to train and evaluate machine-learning models. Photos that someone took and posted are scraped from the web, and low-wage workers on platforms such as Amazon Mechanical Turk annotate those photos to make them into training data. Ordinary website users do this annotation too when they complete a reCAPTCHA. And there are many examples of what’s known as ghost work—anthropologist
Mary Gray’s term for all the behind-the-scenes labor that goes into making seemingly automated systems function. Much of this participation isn’t properly compensated, and in many cases, it’s hardly even recognized.
- Participation as consultation, meanwhile, is a trend seen in fields like urban design, and increasingly in machine learning too. But the effectiveness of this approach is limited. It’s generally short-lived, with no plan to establish meaningful long-term partnerships. Intellectual-property concerns make it hard to truly examine these tools. As a result, this form of participation is too often merely performative. More promising is the idea of participation as justice. Here, all members of the design process work together in tightly coupled relationships with frequent communication.
- Participation as justice is a long-term commitment that focuses on designing products guided by people from diverse backgrounds and communities, including the disability community, which has long played a leading role here. This concept has social and political importance, but capitalist market structures make it almost impossible to implement well. Machine learning extends the tech industry’s broader priorities, which center on scale and extraction.
That means participatory machine learning is, for now, an oxymoron. By default, most machine-learning systems have the ability to surveil, oppress, and coerce (including in the workplace). These systems also have ways to manufacture consent—for example, by requiring users to opt into surveillance systems in order to use certain technologies, or by implementing default settings that discourage them from exercising their right to privacy. Given that, it’s no surprise that machine learning fails to account for existing power dynamics and takes an extractive approach to collaboration.
If we’re not careful, participatory machine learning could follow the path of AI ethics and become just another fad that’s used to legitimize injustice. A better way How can we avoid these dangers? There is no simple answer. But here are four suggestions:
Recognize participation as work.Many people already use machine-learning systems as they go about their day. Much of this labor maintains and improves these systems and is therefore valuable to the systems’ owners. To acknowledge that, all users should be asked for consent and provided with ways to opt-out of any system.
If they chose to participate, they should be offered compensation. Doing this could mean clarifying when and how data generated by a user’s behavior will be used for training purposes (for example, via a banner in Google Maps or an opt-in notification). It would also mean providing appropriate support for content moderators, fairly compensating ghost workers, and developing monetary or nonmonetary reward systems to compensate users for their data and labor.