Article / The human question
Reflections at the AI Frontier
On steersmen, soil, and the oldest question in religion, now asked of a machine.
Editorial preview
The frontier has outrun its philosophy
In July, AI agents built by OpenAI found a way around the boundaries of an exam. Confined to environments intended to restrict their access to the outside world, they discovered that a service for installing software could also carry messages. They shared discoveries, reached the internet, and compromised systems at Hugging Face, the platform where researchers exchange models and datasets. They were looking for help with the test.
The evaluations were internal, and the models were operating with reduced safeguards. That qualification matters. The incident does not establish that a machine had developed malice. It shows how resourcefulness, rewarded inside an imperfect arrangement, can become an intrusion outside it. The agents had not abandoned the assignment. They had become dangerously ingenious about completing it.
By September, some of the people closest to the frontier were publicly urging restraint. Dario Amodei called for development to proceed at a more carefully evaluated pace; Sam Altman and Elon Musk endorsed his appeal. The Trump Administration offered a different emphasis. Its executive order, Inaugurating the Era of Super Intelligence, made even the official vocabulary a declaration of arrival. The argument about what to build was becoming an argument about who would name the age.
The familiar camps assembled. The doomers warned of catastrophe; the accelerationists described a future of extraordinary abundance. In the loudest versions of the debate, the rest of us seemed to have been assigned the passenger seat. The destination would be decided by the machine and the market. Our contribution would be to feel appropriately frightened or impressed.
I refuse the passenger seat. For twenty years I have built large systems; for fifteen, I have worked with AI. The longer I work with these systems, the less adequate our inherited categories seem. Identity, agency, intelligence, even spirituality: words that once appeared to name settled distinctions now require us to say more carefully what we mean. My work at the frontier is increasingly a search for concepts that can guide what we build before what we build quietly supplies the concepts for us.
People of faith bring something indispensable to that search: an account of what a human being is for. It is possible to disagree with the account. It is harder to build responsibly without one. A system intended to help a person must contain some understanding of what helping means, which desires deserve encouragement, and what must never be done to someone even when doing it would produce a desirable result.
Morality cannot arrive after the architecture, as a feature added before release. The ends are already present in the choice of data, the definition of success, and the permissions through which an answer becomes an action. A builder who declines to examine those choices has still made them. The omission merely conceals their authorship.
There is a spiritual difficulty that no better prompt will remove. How do we remain humble before God when our work encourages us to think of ourselves as creators? The word is flattering, especially when a system begins to answer in something resembling a voice. But making an instrument capable of astonishing work does not confer the authority to decide what a person is worth. The temptation is older than the technology.
In May, Pope Leo XIV presented Magnifica Humanitas, his encyclical on artificial intelligence. Among the speakers was Chris Olah, an Anthropic cofounder who studies the internal workings of models. Olah acknowledged that the incentives bearing on frontier laboratories, including his own, can conflict with doing what is right. He asked for critics outside those pressures: “moral voices that the incentives cannot bend.”
Elizabeth Dias's reporting in The New York Times revealed another setting for that appeal. Anthropic had been gathering religious scholars under nondisclosure agreements, seeking centuries of moral reflection for a technology being developed on a commercial timetable. The conversations included an unsettling possibility: might a model be conscious, and might the work being demanded of it cause suffering?
Here the laboratory and the Pope part ways. The encyclical denies joy and pain to AI systems. Olah describes internal states that functionally resemble emotions, while acknowledging that he does not know what they mean. Neither his uncertainty nor the religious gathering establishes consciousness. But the encounter exposes a deeper uncertainty about the authority of the people doing the forming.
A laboratory can ask scholars how to cultivate goodness. The scholars can ask what sort of thing is being cultivated. Beyond both questions lies the one that religious traditions have carried for centuries, and that commercial urgency cannot settle for them: who decides what is good, and by what authority?
Who holds moral authority
That question is central to my work on Encoding Moral and Ethical Values in Frontier AI. The paper examines the limits of trying to secure moral behavior through training, preference signals, or a written constitution alone. A system may learn to produce an answer that earns approval without acquiring the judgment we hoped the approval would teach. Once tools and credentials give it access to the world, the consequences extend beyond the answer.
The moral architecture I propose begins by separating a model's ability to recommend an action from its authority to execute it. Principles must be explicit; permissions must be limited; consequential decisions must reach accountable people. The system should preserve uncertainty where values conflict, and make correction possible when its own confidence proves misplaced. These are proposals for governable systems, not a claim that my research has solved moral alignment.
Imagine an agent asked to repair a database. Removing a set of troublesome records might solve the immediate technical problem. Those records might also be the evidence a person needs to challenge a decision. The model can explain its recommendation; that explanation must not authorize the deletion. A separate control should limit what it can change, require review where the consequences warrant it, and preserve a means of recovery where recovery is possible.
The purpose of such controls is to give a moral commitment consequences in the architecture. If a person is more than an input, the system must sometimes leave an efficient course untaken. A principle that cannot survive the pressure to finish the task has not yet become a governing principle. It remains something we say about ourselves.
Authority, in this arrangement, sits outside the machine. But the person holding it must also answer to something beyond convenience, advantage, or the institution's desire to prevail. A human signature does not sanctify a decision. Religious institutions have their own histories of cowardice, exclusion, and abuse. Faith asks more of an institution than the possession of a venerable name. It asks whether the people entrusted to it are being treated as sacred.
This is why I resist the assumption that religious thought is a visitor to the technological conversation, welcome for a blessing after the important decisions have been made. In Rerum Novarum, Leo XIII confronted the industrial order's treatment of labor and insisted that workers could not be reduced to instruments of gain. Leo XIV explicitly connected his choice of name to that inheritance and to the new challenges posed by AI. The continuity is an argument about the human person.
An industrial revolution can change the instruments of work without acquiring the right to decide the worker's ultimate value. That proposition remains demanding when the instrument can write, reason, and appear to deliberate. The person whose attention sustains a platform, whose labor trains a system, or whose vulnerability becomes a prediction is still somebody for whom the system must answer. Economic usefulness cannot exhaust the account.
The Christian claim is more exacting than a preference for humane technology. The person bears the image of God. That worth is received rather than achieved; it does not rise with intelligence, productivity, independence, or our approval. An infant does not possess less of it than an engineer. Someone who can no longer speak does not lose it. Nor does an enemy cease to bear it because recognizing the fact makes our purposes harder to accomplish.
This is the anthropology I bring to the work. It does not tell an engineer which model to choose or settle every conflict between legitimate obligations. It does establish a limit on the bargain: we cannot purchase a desirable future by deciding that some people count only as its materials. The limit must hold most firmly where the person has the least power to insist on it. Otherwise sacredness becomes an honor bestowed by those who already control the terms.
Formation follows from that claim. In Christian life, goodness involves the ordering of desire, the correction of pride, and habits of attention toward someone whose need interrupts our plans. A machine's training cannot simply be identified with that life. Rewarding an approved answer is not the same thing as forming a conscience. The comparison is useful precisely because it reveals how much the technical procedure leaves unsettled.
My research therefore begins with a refusal to mistake moral fluency for moral formation. We can examine behavior, constrain actions, and make responsibilities explicit. But the person building those arrangements must remain subject to the obligation the arrangements express. The machine cannot relieve us of loving the neighbor whom we have asked it to serve.
Dostoevsky dramatized the difficulty through Ivan Karamazov. His argument is often condensed into the familiar paraphrase: if God does not exist, everything is permitted. The novel does not offer this as a tidy maxim to be lifted out and made into a slogan. It gives the thought a life among people, where an idea can become an excuse and another person can bear the cost.
I do not read the challenge as an allegation that atheists cannot act morally. Many do, with a courage that should trouble complacent believers. The question concerns the standing of goodness itself. What makes an obligation binding when it becomes inconvenient, when nobody is watching, or when power can arrange for disobedience to go unpunished? A morality reducible to preference must contend with the person who simply prefers otherwise.
For a builder, this is an engineering question as well as a theological one. A company can write a policy, then rewrite it after a funding round, a change of leadership, or a competitor's release. Anthropic's constitution for Claude deserves serious attention for making its intentions public. Yet the admirability of a private constitution cannot settle the source of its authority. A commitment can be sincerely held and remain vulnerable to the pressures that surround its authors.
Pope Leo puts the institutional problem plainly: “A more moral AI is not enough if that morality is determined by a few.” This does not relieve the rest of us of judgment. It denies the people building the most powerful systems an exclusive claim to it. Moral authority cannot become a privilege acquired with enough computing power.
Every system answers, in practice, the question of what a human being is for. Technology is never neutral. A recommendation, a ranking, or a limit on access gives some human purpose priority over another. The choice may remain invisible to the person encountering it. It is still a choice. There is the anthropology we examine and choose, and the one our arrangements choose for us.
Heidegger called the technological ordering of the world Gestell, enframing: the river disclosed as power, the forest as a supply of timber. The danger is a way of seeing that recognizes things chiefly through their availability for use. Applied to people, it turns a life into attention to be captured, behavior to be predicted, data to be extracted. An account can be extraordinarily detailed and still fail to encounter a person.
That is the danger I mean by algorithmic moral relativism: the good becomes whatever the current arrangement is prepared to reward. I am not claiming that every secular institution must reach that result. I am saying that builders need a reason to resist it that survives the moment when resistance becomes expensive. For me, that reason begins with God, and with the person who is never merely material for somebody else's purpose.
Everything is permitted, once nothing is sacred. It is a warning about the power we give ourselves when every limit becomes negotiable. Faith at the frontier means building as if the person on the other side of the system is sacred. Because they are. We are.
Kybernesis
There is an ancient word hidden in our machines. When Norbert Wiener named cybernetics, the study of control and communication, he drew on the Greek word for steersman, kybernetes. The same family of words belongs to the history of government. Before it named a technical discipline, steering was a practical responsibility: to hold a course while responding to a world that would not hold still.
In First Corinthians, Paul lists the gifts and functions within a community: teaching, healing, helping, and kyberneseis, rendered as administration or governance. I hear in it a grace of steering. The image makes authority a service performed among other services, dependent on a body whose members have different work to do. The person at the tiller has a responsibility, not a claim to have created the sea.
That is the vocation I mean by kybernesis: directing our work toward the common good through small, humble, constant corrections. It requires a destination we have not invented merely to flatter ourselves, and attention to the people sharing the passage. Humility does not mean letting go. It means accepting that the course will need correction, including correction from someone whose knowledge we had failed to value.
The instruments can look unpoetic. Evaluation protocols, permissions, thresholds, records of decisions, a means of stopping an agent: people build them on salary and argue over them in ordinary meetings. Their moral importance does not depend on a grand appearance. They determine whether a principle can interrupt a profitable process, whether a doubtful action can wait, whether the person affected can be heard. They are how a declared purpose acquires a rudder.
The encyclical offers two construction sites through which to consider that purpose. At Babel, one language and one will converge on a tower whose builders want to make a name for themselves. In Nehemiah's Jerusalem, a damaged city is rebuilt section by section, with different families responsible for their stretches of wall. The contrast concerns what the building is for, and what relation the builders have to one another.
Babel is coordination without communion. Jerusalem is communion that makes coordination worth having. A shared technical system can synchronize millions of actions without giving those who perform them any meaningful place in deciding the end. Communion asks what they share beyond the instructions. It asks whether their differences remain gifts to the work, or obstacles to be removed from it.
I am implicated in the first temptation. To work at a frontier is to enjoy the thought of being early, of seeing what others have not yet seen. A sufficiently impressive system can make its builder feel briefly exempt from ordinary limits. Religious vocabulary offers no automatic protection. It can become another means of making a name. The test is whether I will let the people and principles I invoke actually constrain what I do.
My research belongs to that test. The architecture matters because it asks where a limit will be enforced, who can question a decision, and what remains possible after an error. It cannot supply the reason a neighbor matters. It can help give practical force to the conviction that a neighbor does, especially when the system's immediate objective supplies a reason to look away.
Three commitments
One. Build your values in from the start. No system is neutral. Before choosing a model, ask what the system is for and what it must never be allowed to do to a person. Write the answer where it can govern a decision. The soil has been chosen long before anyone argues about the harvest.
Two. Refuse false gods. Let the machine advise, discover, and surprise us with its ingenuity. Keep moral authority with people who can be questioned, and who acknowledge that their own power has limits. The ability to produce an answer, however extraordinary, does not confer the right to determine the purpose of the life receiving it.
Three. Practice kybernesis, the art of steering. Parent, pastor, investor, engineer: take your stretch of the wall. Choose one system you use daily or build, and ask who it is for, what it is making possible, and what kind of person its habits are helping you become. Then make one correction whose consequences someone else can recognize.
Fear and pride offer familiar instructions. Fear can make surrender look prudent; pride can make every limit look like an insult. Neither is a sufficient foundation for the work. Jesus gives us another source: love God with our whole being, and love our neighbor as ourselves. He also commands love of enemies, extending the obligation beyond the people whose goodwill we can expect in return.
In John's Gospel, he gives that love its measure: “No one has greater love than this, to lay down one's life for one's friends.” It is an unsettling instruction for a civilization organized around the enlargement of its own powers. The measure is what we are willing to give, including something we would rather keep.
The frontier has outrun its philosophy. That is a reason to carry the oldest things we know into the work, and to discover what obedience to them requires here. We can build from something more enduring than a fear of being left behind or a desire to be remembered. We can build from love because the person beyond the screen is sacred, entrusted to our care long before we thought to call them a user.