Can AI be moral? The quest for moral machines and what it means for human morality
Yuxin Liu, CTMF PhD Fellow
Can machines be taught to understand right from wrong? Should we build AI that can navigate complex ethical dilemmas, or is morality an inherently human trait that technology can never truly grasp? How do we design AI with moral capacity? These are some of the key questions in machine ethics – a field dedicated to equipping AI with moral reasoning abilities in the pursuit of artificial moral agents (known colloquially as moral AI).
The concept of a moral AI is broader than it seems. Technically, even a toaster counts as one in a very weak sense: after all, it may accidentally burn someone and cause harm. But the ultimate goal of machine ethicists is more ambitious – to create full ethical agents that can understand moral principles, adapt to context, and justify their decisions.
Unsurprisingly, the long-term goal of such an advanced system remains far-fetched and current efforts have switched to more limited forms, such as Artificial Moral Advisors (AMAs). Unlike fully autonomous moral agents, AMAs function as an advisory tool to provide moral guidance based on embedded ethical frameworks that are deemed acceptable by programmers, ethicists, and/or the general public. The idea is that they could help us make better moral choices and even cultivate better moral motivations and characters – a process philosophers call AI moral enhancement.
General-purpose AI systems like ChatGPT can be regarded as a form of AMA, as they can offer moral advice when prompted. There are also more specialised proposals: for example, Giubilini and Savulescu suggested an AMA that allows individual input of moral preferences as guiding criteria to offer personalised best courses of action. AskDelphi is another real-world example: a research prototype chatbot developed by the Allen Institute that offered simple "yes" or "no" answers to users' moral dilemmas. Despite its popularity upon release, AskDelphi faced criticism from scholars and the public for its Western-centric viewpoints of its training dataset, at times offensive or absurd judgments, and inability to adapt to shifting moral norms.
Many have pointed out other complications in AMAs providing prescriptive moral advice, including the risk of over-reliance without users critically engaging in moral reflection themselves. To address this, some scholars advocate for an alternative, dialogic approach inspired by the Greek philosopher Socrates: a Socratic AI that, rather than telling people what to do based on pre-designed principles, would engage its users in back-and-forth deliberative exchanges to facilitate moral reasoning.
A small step in this direction is the Peter Singer AI (PSai), a chatbot trained on Singer’s writing to discuss topics such as global poverty, veganism, and animal rights. Whilst PSai can serve as a valuable entry point for exploring moral philosophy, upon closer engagement, users have observed its wobbly stance on thorny issues and lack of sophistication compared to a real dialogue with Singer. At best, its attempts at prompting reflection are limited to a somewhat generic follow-up question at the end of every answer. Unlike Socrates, who insisted on challenging commonly-held assumptions and exposing inconsistencies in others’ arguments, PSai does not possess the moral sensitivity required to pick up nuances in conversations that could spark deeper deliberation, nor does it help the user discover contradictions or other gaps in their own reasoning. Without genuine moral understanding, future moral AI would likely face similar challenges as PSai.
Where does this leave us? Given the irreducibility and inescapability of human moral judgement and responsibility, AMAs could perhaps serve as an informational tool to help fill the gaps in our knowledge or expertise. However, once the initial fascination fades, deeper questions emerge: are AMAs a viable and effective solution if our responses to them remain subject to the same moral imperfections of our moral psychology that they are designed to fix? And more importantly, is the quick turn to technology for moral remedies truly desirable, or a distraction from confronting and addressing the root sources of ongoing moral, ethical, environmental, and political crises? Before moving forward, we must critically examine the motivations behind the pursuit of moral machines.
Dive deeper into this topic by reading Yuxin’s extended blog here!
*For references, please see full blog.
About the contributor:
Yuxin Liu is a PhD Fellow at the Centre for Technomoral Futures. Her current research involves the interdisciplinary study of moral psychology, including thinking and reasoning, cognitive biases and heuristics, moral intuitions, moral decision-making, and human-AI interaction. Her project, ‘Human Moral Judgements Towards Artificial Intelligence Systems’, is co-supervised in the School of Philosophy, Psychology and Language Sciences by faculty in Philosophy and Psychology.