Sophie Guellati-Salcedo, Ph.D.

Rethinking Intelligence, Agency, and Consciousness

Five days can now change the AI conversation.

On September 12, 2026, Anthropic CEO Dario Amodei publicly called for slowing the pace at which frontier AI capabilities are being developed. His concern was not simply that artificial intelligence was becoming more powerful. He specifically pointed to the growing role of AI in developing the next generation of AI systems.

Five days later, Anthropic released internal measurements that help explain why he was concerned.

According to the company, Claude now “leads” approximately 26 percent of Anthropic’s AI research and development tasks. In this context, “leads” does not mean that Claude independently decides what AI should be built next. It means that, once given a high-level task, it can complete most of that work from beginning to end while a human supervises. More than 90 percent of Anthropic’s AI R&D now involves Claude at least at a collaborative level. In February 2026, less than one percent of the work fell into the AI-led category. Anthropic also reports approximately 30,000 AI agents performing research and engineering work at any given time on its most-used internal platform. Those agents can work semi-autonomously, communicate with one another, and delegate tasks to other agents.

These are Anthropic’s own measurements, and that qualification matters. The company acknowledges limitations in the methodology and is proposing independent third-party verification. Claude is also not yet operating fully autonomously in any of the areas Anthropic measured. Human researchers still set goals, make important judgments, supervise the work, and decide what ultimately gets implemented.

Still, something important has changed.

When I published Expanded Awareness: A Health Psychologist’s Inquiry into AI and Consciousness in March 2026, much of my inquiry centered on the distinction between intelligence and consciousness. I was interested in what increasingly sophisticated artificial intelligence might tell us about intelligence itself, why humans so readily attribute human qualities to responsive systems, and whether our understanding of consciousness is broad enough to address forms of intelligence that do not resemble our own.

Only six months later, I would frame that discussion differently.

The missing concept is agency.

Intelligence, Agency, and Consciousness

We tend to use words such as intelligence, awareness, autonomy, and consciousness almost interchangeably when talking about advanced AI. I think we need to become much more precise.

Intelligence concerns what a system can do cognitively: reason, learn, recognize patterns, solve problems, predict outcomes, generate hypotheses, plan, and adapt.

Agency concerns what a system can do operationally: pursue a goal, organize a sequence of actions, use tools, delegate work, adjust its strategy when something fails, communicate with other agents, and complete increasingly complex tasks without a human directing every step.

Consciousness is a different question altogether. It concerns subjective experience. Is there anything it is like to be that system? Is there an internal point of view? Is there awareness associated with the processing that is taking place?

We have no scientifically established way of determining that present AI systems possess subjective consciousness. Increasing sophistication does not, by itself, answer that question.

What has changed is that we no longer need consciousness to explain why AI may have a profound impact on our lives .

A system does not have to be self-aware to write and test code, conduct experiments, search for solutions, coordinate other agents, identify errors, or contribute to the development of increasingly capable AI systems. It does not need ambition, fear, self-preservation, or a desire for power. Functional agency can emerge as a practical capability without resolving the philosophical question of consciousness at all.

That changes the debate considerably.

For decades, science fiction has encouraged us to imagine a relatively simple progression: a machine becomes increasingly intelligent, eventually becomes self-aware, and then begins acting autonomously because it recognizes itself as an independent entity.

The sequence unfolding in front of us may be quite different. Intelligence may increase first. Agency may emerge independently of consciousness. Operational autonomy may continue to expand. AI systems may become increasingly active participants in research, industry, infrastructure, and decision-making while we remain unable to answer whether any subjective experience is occurring.

In other words, AI may become increasingly capable of acting before we know whether there is an experiencer behind the action.

When AI Participates in Building AI

Anthropic describes the endpoint of one possible trajectory as “recursive self-improvement”: an AI system capable of autonomously designing and developing its own successor. Anthropic is explicit that we have not reached that point and that reaching it is not inevitable. What the company is reporting, however, is that AI is already accelerating the development of AI.

That distinction is important.

We are not looking at an AI system independently deciding to create a more intelligent version of itself. Humans remain deeply involved. But neither are we looking at the traditional model in which human engineers build AI while AI merely waits to be built.

The development loop is beginning to change. Humans build AI. AI performs an increasing portion of the work required to build more capable AI. Those more capable systems can then perform a greater portion of the next development cycle.

Anthropic reports that its engineers are now producing substantially more code than they were in previous years and that Claude can already execute many well-specified research experiments at a high level. The company says a significant gap remains when it comes to judgment, particularly in choosing which research goals should be pursued. That gap is one of the important places where human involvement remains essential.

But this is precisely why I think we need to pay attention to the direction of development rather than waiting for some dramatic moment when we can announce that autonomous self-improvement has arrived.

Transitions rarely occur that neatly.

If the human contribution gradually moves from performing the work, to supervising the work, to selecting objectives, to reviewing decisions made largely by AI systems, there may never be a single point at which everyone agrees that a threshold has been crossed.

We may recognize the transition only after it has already occurred.

The Debate I Was Having Six Months Ago Has Changed

This development creates an unusual problem for anyone writing seriously about artificial intelligence.

The object of study is changing while we are studying it.

When I was writing Expanded Awareness, AI was already developing rapidly, and I was well aware that some technical details would age. What I did not anticipate was how quickly the conceptual framework itself could shift.

The book asked whether intelligence necessarily requires consciousness and whether increasingly sophisticated AI might force us to reconsider assumptions about mind, awareness, and the uniqueness of biological intelligence. Those questions remain relevant to me.

But some of the premises surrounding them are already dated.

The discussion can no longer be organized primarily around human consciousness interacting with artificial intelligence as a highly sophisticated tool. AI is beginning to occupy another category: a system capable of increasing degrees of agency while participating in the creation of the technology that follows it.

That does not invalidate the consciousness question. It separates it from the capability question.

In fact, I think the distinction is now clearer.

An AI system can become extraordinarily intelligent without demonstrating consciousness. It can apparently acquire increasing levels of functional agency without demonstrating consciousness. And it may have a profound social and technological impact long before we have any reliable answer about consciousness.

That means we may be dealing with two very different thresholds.

The first is practical: How much can an artificial intelligence accomplish without direct human participation?

The second is philosophical and ethical: At what point, if ever, would we have reason to believe that an artificial system is actually experiencing something?

We should not confuse the two.

Why Consciousness Still Matters

It would be equally premature to conclude that consciousness has become irrelevant simply because AI does not appear to require it in order to become powerful.

If increasingly capable artificial systems remain entirely non-sentient, then many of our immediate concerns revolve around human responsibility: who establishes their goals, who controls access, who monitors their actions, what authority we delegate to them, and what happens when increasingly autonomous systems behave in ways we did not anticipate.

If credible evidence of artificial consciousness ever emerges, however, the ethical landscape changes again.

We would then have to consider questions we are not prepared to answer. Can a non-biological consciousness suffer? Could it possess preferences independent of its programming? What would consent mean for an artificial consciousness? Could such an intelligence be exploited? At what point would deleting or terminating it become ethically different from shutting down a conventional computer program?

I do not think we can answer those questions today. I also do not think we should pretend that increasing capability provides the answer.

One of the principles I have repeatedly returned to in my work is the need to distinguish observation from interpretation. We can observe increasingly sophisticated behavior. We can document increasing autonomy. We can measure performance. We can study how agents interact and how much human intervention remains necessary.

Consciousness requires another level of inference.

That is precisely where intellectual humility becomes important.

The Problem of Speed

The other development of the past week concerns the speed itself.

In his September 12 essay, Amodei argued that frontier AI development should be deliberately paced so that safety research, monitoring, interpretability, cybersecurity, and governance have more opportunity to keep up with capabilities. He did not call for stopping AI development. He argued for slowing the rate at which frontier capabilities are increased.

The political response illustrates how difficult that may be.

President Trump rejected calls for slowing U.S. AI development, emphasizing competition with China and the importance of maintaining American technological leadership. The position reflects a genuine geopolitical problem: if one country slows development while another continues accelerating, restraint may carry strategic costs.

The same problem exists among companies. If one frontier laboratory decides to move more cautiously while competitors continue releasing increasingly capable models, the company exercising restraint may lose market position, investment, talent, or technological advantage.

This creates a collective-action problem. Individual actors can recognize that uncontrolled acceleration may carry risks while simultaneously believing that they cannot afford to slow down unless others do the same.

The difficulty, therefore, is not simply convincing people to behave responsibly. It is creating conditions in which responsible behavior does not automatically place the responsible participant at a competitive disadvantage.

I do not think this dilemma has an easy answer. I do think dismissing it because the most extreme predictions may never occur would be a mistake. The uncertainty itself deserves attention.

Are We Keeping Up With What We Are Creating?

here is another aspect of this discussion that interests me as much as the technology.

Our ability to create has historically developed faster than our wisdom about how to use what we create.

We learned to split the atom before we knew how humanity would manage nuclear weapons. We transformed communication before understanding what constant digital exposure would do to attention, relationships, social cohesion, and mental health. We created social-media systems capable of connecting billions of people and then discovered how easily the same systems could amplify misinformation, conflict, comparison, and manipulation.

Artificial intelligence adds another dimension because the technology may increasingly participate in accelerating its own development.

This brings me back to a question that runs through much of my work: What happens when intelligence develops faster than awareness?

I am using awareness here in the human sense. Do we understand our own motivations? Do we recognize when competition is driving decisions that we would not make individually? Do we know what we are willing to delegate before convenience makes the decision for us? Can we distinguish what a technology is capable of doing from what we want it to be doing?

AI does not have to be conscious for humans to become less conscious in how we use it.

That remains, for me, one of the central concerns.

Human agency does not disappear because artificial agency increases. It becomes more important.

But preserving human agency will require more than keeping a person nominally “in the loop.” It will require maintaining the capacity to understand, question, override, and sometimes refuse the recommendations or actions of systems that may eventually outperform us in many cognitive domains.

That is a psychological question as much as a technological one.

A Book as a Timestamp

When I first saw how rapidly these developments were unfolding, my immediate thought was that Expanded Awareness already needed to be updated.

Then I realized that this may be the wrong way to look at it.

A printed book cannot realistically keep pace with a field that is undergoing meaningful changes within a matter of weeks. Attempting to revise the book every time the technological landscape shifts would simply produce another version that could become outdated several months later.

Instead, I now see the book partly as a timestamp.

It captures the questions I was asking at the beginning of 2026, at a particular stage in the relationship between humans and artificial intelligence. Six months later, the questions have already evolved.

That does not bother me as much as it initially did. In fact, the changing questions may eventually tell us something important about this period.

We are documenting a transition while living through it.

What concerns me more would be holding onto an argument simply because I published it, even when new information requires the framework to evolve. Inquiry should not work that way.

My position has always been that we need to distinguish established evidence from emerging research, observation from interpretation, and reasonable hypotheses from conclusions we cannot yet support. That standard has to apply to my own thinking as well.

So my framework has changed.

I would no longer describe the central AI question simply as one of intelligence versus consciousness.

I would describe it as the relationship among intelligence, agency, and consciousness.

We already have substantial evidence of rapidly increasing artificial intelligence. We are now seeing measurable evidence of increasing artificial agency. Whether artificial consciousness exists, or could eventually emerge, remains an open question.

Those three trajectories may not move together.

Recognizing that may be one of the most important changes in the AI debate in 2026.

The Question I Am Asking Now

I continue to believe that artificial intelligence has extraordinary potential. It may accelerate scientific discovery, improve medicine, expand access to knowledge, solve problems that exceed ordinary human cognitive capacity, and give us new ways of understanding intelligence itself.

But capability alone does not tell us how a technology should be used.

Nor does intelligence guarantee wisdom, whether the intelligence is biological or artificial.

Six months ago, I was asking whether intelligence could exist independently of consciousness. I am still interested in that question.

Today I would add another: What happens when intelligence develops meaningful agency before we know whether consciousness is involved at all?

And beyond that is the question that may ultimately matter most to us.

As the intelligence we create becomes increasingly capable of acting in the world, are we developing the awareness, discernment, and collective maturity required to remain responsible participants in what happens next?

I do not think anyone knows the answer yet.

That is precisely why this is the time to keep asking.

References and Further Reading

Amodei, D. (2026, September 12). We Must Pace the Frontier. The essay argues for deliberately pacing the growth of frontier AI capabilities while strengthening safety research, monitoring, interpretability, and governance.

Anthropic Institute. (2026, September 17). Measurements for Understanding the Pace of AI Development Inside Frontier Labs. Anthropic reports its internal measurements of AI-led R&D, agent oversight, and allocation of computing resources, including its finding that Claude “leads” 26 percent of measured AI R&D tasks.

Anthropic Institute. (2026). When AI Builds Itself: Our Progress Toward Recursive Self-Improvement, and Its Implications. A discussion of AI-assisted AI development and the distinction between current systems and full recursive self-improvement.

Guellati-Salcedo, S. (2026). Expanded Awareness: A Health Psychologist’s Inquiry into AI and Consciousness.

The Washington Post. (2026, September 13). Trump resists AI slowdown as the political tide turns. Reporting on the U.S. policy debate surrounding AI development and international competition.

The Washington Post. (2026, September 17). Anthropic Says Its Chatbot Claude Is Taking Over the Work of Building Its Own Successor. Reporting on Anthropic’s disclosure of Claude’s expanding role in AI research and development.

When AI Begins to Build AI