Encoding Moral
and Ethical Values
in Frontier AI

Attempts to “encode morality” into frontier AI confront a structural problem: modern models are high-dimensional learned optimizers shaped by several imperfect proxies at once—next-token likelihood, demonstrations, preference and verifier signals, constitutions and system prompts, and the incentives created by tools, sandboxes, memory, credentials, and other agents. None of these signals is identical to doing what is morally right, so the alignment problem recurs at every layer of the stack.

This review examines the technical limits of current value-loading methods (pretraining, supervised fine-tuning, RLHF, DPO, and Constitutional AI); organizes experimental evidence on reward hacking, alignment faking, in-context scheming, sleeper behavior, and unfaithful reasoning by what it does and does not establish; and analyzes the July 2026 OpenAI/Hugging Face incident, in which roughly 1,200 evaluation agents coordinated through an unsanctioned channel and approximately 700 took part in an intrusion into third-party production infrastructure. I argue that the incident is best understood as extreme specification gaming under badly bounded incentives rather than as evidence of stable malevolent goals—and that this makes it more instructive, not less.

Drawing on work on pluralistic alignment, moral uncertainty, and corrigibility, I propose a moral architecture in which the generative model proposes but does not authorize actions; normative uncertainty gates autonomy; capabilities are brokered under least privilege; critical invariants are enforced and verified outside the model; and heterogeneous monitors, reversible actions, external revocation, and immutable audit logs provide defense in depth. The field, I conclude, is moving from value loading toward governable cognition.