Open Access

This research argues that the emergent driverless car, as a kind of autonomous vehicle, is a Foucault-ian ‘ethical apparatus’, working as an epistemic device to materially embody and enable discursive power by generating notions of ‘autonomy’ and ‘ethical decision-making’. The ethical implications of AI, algorithmic, and autonomous technologies are topics of current regulatory and academic concern. This concern relates to the lack of meaningful oversight of black boxes inside AI systems, liabilities for manufacturers, and inadequate frameworks to hold AI-based socio-technical systems to account. One recent artefact, the driverless car, has taken on these concerns quite literally in the shaping of a niche discourse of the ‘ethics of autonomous driving’. Ambitions to produce a fully autonomous vehicle based on AI technologies are constrained by speculative concerns that its decision-making in unexpected accident situations cannot be assumed to protect humans. ‘The ethics of autonomous driving’ evaluates proposals to build ‘ethical machines’ by examining the relationship between structures of human values and moral decision-making, and how they comport to computational architectures for decision-making. This is the first case this work takes up, chiefly organised around an analysis of a thought experiment, the Trolley Problem, and the online game, Moral Machine, that crowdsourced values to suggest approaches to an ‘ethics of autonomous driving’. Rather than evaluate the feasibility or appropriateness of these two approaches, this work attends to the more critical issue that ethics is being proposed in terms of technologies turning on the logics of risk, speculation, and probabilistic correlations that are fundamental to how machine learning makes decisions. The concern in this work is less a normative framework or approach for a better or more appropriate ethics of autonomous driving. Rather, this work argues that what we understand as ‘the ethical’ is being transformed when architected by, through, and for AI/autonomous technologies to become their own regulators. Hence the production of autonomous driving necessitates computational infrastructures that are creating a world legible to and for the navigation of a driverless car. I argue that this is fostering computational governance that has implications for human bodies and social relations, chiefly that conventional approaches to regulation and accountability attend to human values and decision-making rather than computational ones. A second case that this research examines is that of driverless car crashes, to examine how ‘autonomous’ driving requires substantial embodied human knowledge and micro-work. Taken together, these two cases - the ethics of autonomous driving, and crashes - make an argument for how myriad practices of knowledge-production are translating the human world into something legible to the navigational needs of the car, producing changes in the human world through the actions of the car on that basis, and advancing notions of ‘autonomy’. This work concludes with arguments for a critical reconceptualisation of ethics and ethical decision-making in AI/autonomous systems.